GoSpider
Created by Unit 515High-Performance Web Crawler for Security Reconnaissance
GoSpider is an open-source web crawler designed to automate web reconnaissance, discover endpoints, and extract relevant information from web applications.
- Concurrent crawling with depth control
- JS, sitemap and robots.txt parsing
- Subdomain and AWS S3 bucket extraction