Cyberchef Recipes

Extract a list of unique URLs based on the input

Accept one URL per line, filter the output list so that we only get URLs with the same domain as the request, tries to alter relative URLs to absolute URLs so the operation Extract URLs works correctly.

You Need to run chrome with the following arguments –disable-web-security –user-data-dir=”path_to_profile”, don’t use your day to day browser profile, create a new one just for cyberchef

[
  { "op": "Fork",
    "args": ["\\n", "\\n\\n", false] },
  { "op": "Register",
    "args": ["(.*)", true, false, false] },
  { "op": "Extract domains",
    "args": [false, false, false] },
  { "op": "Register",
    "args": ["(.*)", true, false, false] },
  { "op": "HTTP request",
    "args": ["GET", "$R0", "", "No CORS (limited to HEAD, GET or POST)", true] },
  { "op": "Conditional Jump",
    "args": ["content-type: text/html;", true, "END_FORK_LOOP", 1] },
  { "op": "Find / Replace",
    "args": [{ "option": "Regex", "string": "((href=\\\"\\/)|(src=\\\"\\/))" }, "https://$R1/", true, true, true, false] },
  { "op": "Extract URLs",
    "args": [false, true, true] },
  { "op": "Filter",
    "args": ["Line feed", "$R0", false] },
  { "op": "Label",
    "args": ["END_FORK_LOOP"] },
  { "op": "Merge",
    "args": [true] },
  { "op": "Extract URLs",
    "args": [true, true, true] }
]