Scraping Sites Management
Configure and manage job scraping sources
Active Sites
freelance.de
German Freelance Platform
Configuration:
• URL: https://www.freelance.de
• Login: Required (Premium)
• Remote: Checkbox filter in sidebar
• Project ID: /projekt-{ID} in URL
• Keywords: Skills parameter
FreelancerMap
IT Freelance Platform
Configuration:
• URL: https://www.freelancermap.com
• Login: Required
• Remote: Search filter available
• Project ID: /project-{ID} in URL
• Keywords: Search parameter
Indeed España
Spanish Job Search Engine
Configuration:
• URL: https://es.indeed.com
• Login: Not Required (Public)
• Remote: In job description
• Anti-Bot: Stealth Plugin
• Keywords: Search parameter
Indeed Deutschland
German Job Search Engine
Configuration:
• URL: https://de.indeed.com
• Login: Not Required (Public)
• Timezone: Europe/Berlin
• Anti-Bot: Multilogin + 2Captcha
Indeed France
French Job Search Engine
Configuration:
• URL: https://fr.indeed.com
• Login: Not Required (Public)
• Timezone: Europe/Paris
• Anti-Bot: Multilogin + 2Captcha
Indeed USA
US Job Search Engine
Configuration:
• URL: https://www.indeed.com
• Login: Not Required (Public)
• Timezone: America/New_York
• Anti-Bot: Multilogin + 2Captcha
Indeed Canada
Canadian Job Search Engine
Configuration:
• URL: https://ca.indeed.com
• Login: Not Required (Public)
• Timezone: America/Toronto
• Anti-Bot: Multilogin + 2Captcha
Indeed Australia
Australian Job Search Engine
Configuration:
• URL: https://au.indeed.com
• Login: Not Required (Public)
• Timezone: Australia/Sydney
• Anti-Bot: Multilogin + 2Captcha
Site-Specific Scraping Rules
| Site | Remote Detection | ID Pattern | Auth Required | Rate Limit |
|---|---|---|---|---|
| freelance.de | Checkbox + "Remote" text | /projekt-(\d+) | Premium | 2 sec/request |
| FreelancerMap | Filter options | /project/(\d+) | Required | 2 sec/request |
| indeed.es | Text in description | jk=(\w+) | Public | 10 sec/page |
| indeed.de | Homeoffice/Fernarbeit | jk=(\w+) | Public | 10 sec/page |
| indeed.fr | Télétravail | jk=(\w+) | Public | 10 sec/page |
| indeed.com | Remote/Work from home | jk=(\w+) | Public | 10 sec/page |
| indeed.ca | Remote/Télétravail | jk=(\w+) | Public | 10 sec/page |
| indeed.com.au | Remote/WFH | jk=(\w+) | Public | 10 sec/page |
Adding New Sites
New scraping sites can only be added through code implementation. Each site requires:
- Custom scraper implementation in
/src/scrapers/ - Site-specific selectors and patterns
- Authentication handling (if required)
- Remote job detection logic
- Rate limiting and error handling
Contact the development team to add support for additional job sites.