Search bots generate legitimate Internet traffic, allowing search engines to index your website. But you probably don't want the search bot to trigger fingerprinting. The device fingerprint service maintains its own search engine robot filter list, which you can turn on or off as needed.
Please Note that requests filtered by the search robot will not be returned to blackbox, only relevant error prompts will be returned.
Search robot filtering based on User-Agent is a filtering mechanism that we provide for free to filter out common and well-behaved search bots. Since the detection logic is not complicated and there may be leakage, we will regularly update and maintain the machine filter list.
Configure
1.Log in to the customer portal, navigate to Device Fingerprint- Management-Request Filtering-Bots, and enable Exclude Search Bots
User-Agent Search bots filter list
User-Agent Search bots filter list| Source | Character string | Public Documents |
|---|---|---|
| Mail.ru | mail.ru | Source |
| Microsoft | bingbot AdIdxBot MicrosoftPreview | Source |
| Apple | applebot | Source |
| Huawei | petalbot | Source |
| Naver | naver.me/spd | Source |
| Yahoo! | slurp | Source |
| Baidu | baiduspider | Source |
| Sogou | sogou web spider sogou wap spider sogou pic spider | Source |
| Exsalade | exabot | Source |
| Kokok | coccocbot | Source |
| Seznan | seznamBot | Source |
googlebot AdsBot-Google Storebot-Google APIs-Google AdsBot-Google-Mobile Mediapartners-Google Googlebot-Image Googlebot-News Googlebot-Video AdsBot-Google-Mobile-Apps FeedFetcher-Google Google-Read-Aloud Google-Site-Verification | Source | |
| Yandex | yandex.com/bots | Source |
| Byte Jumping | bytespider | Source |
| Duckduckgo | duckduckbot | Source |
| Ahrefs | AhrefsBot AhrefsSiteAudit | Source |
facebookexternalhit facebookcatalog | Source | |
Pinterestbot | Source | |
| Sincloro | Cincraw | Source |
| Semrush | http://www.semrush.com/bot.html | |
| Sekport | SeekportBot | Source |
| Amazon | Amazonbot/0.1 | Source |
| Data Dog | datadog | Source |