AcadMatchBot: the AcadMatch crawler
Last updated:
What it is
AcadMatchBot collects publicly posted academic research positions from Israeli universities and research institutes (MSc, PhD, postdoc, research staff and similar), so people can find them in one place. Every position on AcadMatch links back to the page it came from.
Listings collected by the crawler are not shown publicly yet.
What it fetches
Only pages we have registered: public pages that list open positions, and the position pages they link to on the same site. It does not sign in, fill in forms, or follow links to other sites.
How it behaves
- It obeys robots.txt, including any Crawl-delay.
- It sends at most one request every 5 seconds to a site.
- It revisits a page about once a day and asks only for changes (conditional requests), so unchanged pages are not downloaded again.
- It identifies itself in every request with a User-Agent containing "AcadMatchBot", a link to this page and orirot13@gmail.com.
How to block it
Add this to your site's robots.txt:
User-agent: AcadMatchBot
Disallow: /Contact and removal
To stop the crawler on your site, report a problem, or ask us to remove a position, write to orirot13@gmail.com. We can hide any position promptly, while keeping an internal record of where it came from.