SnapToolbox LogoSnapToolbox
Back to Snaps

URL Scraper and Separator

The URL Scraper and Separator Snap extracts and copies URLs from the current webpage that match a specified common prefix. This Snap is particularly useful for creating AI-powered knowledge bases using tools like MindPal and Afforai.

How it works

  1. Initiate the Snap: When you activate the Snap, it first captures the current page's URL.

  2. Prompt for Prefix: A prompt appears asking you to enter a common URL prefix. You can either specify a prefix or leave it blank to use the current page's URL as the default prefix.

  3. Collect URLs: The Snap scans all the links on the page, filtering those that match the specified prefix.

  4. Remove Duplicates: It strips out any query parameters or hash fragments from the URLs and ensures there are no duplicates.

  5. Copy to Clipboard: The matching URLs are then joined into a single string, separated by commas, and copied to your clipboard.

  6. Notify User: Finally, it alerts you with the number of URLs copied or informs you if no matching URLs were found.

Use cases

  • Research Projects: Quickly gather all relevant links from a research article or documentation page to compile a list of resources.

  • Content Curation: Collect URLs from a blog or news site that match a specific topic or category for content aggregation.

  • SEO Analysis: Extract URLs from a competitor's site that match certain criteria for analysis and strategy development.

  • Web Development: Gather all links from a development documentation page to ensure all resources are accounted for in a project.

  • Academic Work: Compile a list of references from an online journal or library that match a specific research area.

  • Knowledge Base Creation: Efficiently collect URLs from a site to build a comprehensive knowledge base for AI training or information retrieval.