API Reference
中文

Search bot request filtering

Search bots generate legitimate Internet traffic, allowing search engines to index your website. But you probably don't want the search bot to trigger fingerprinting. The device fingerprint service maintains its own search engine robot filter list, which you can turn on or off as needed.

Please Note that requests filtered by the search robot will not be returned to blackbox, only relevant error prompts will be returned.

Search robot filtering based on User-Agent is a filtering mechanism that we provide for free to filter out common and well-behaved search bots. Since the detection logic is not complicated and there may be leakage, we will regularly update and maintain the machine filter list.

Configure

1.Log in to the customer portal, navigate to Device Fingerprint- Management-Request Filtering-Bots, and enable Exclude Search Bots


User-Agent Search bots filter list

SourceCharacter stringPublic Documents
Mail.rumail.ruSource
Microsoftbingbot AdIdxBot MicrosoftPreviewSource
AppleapplebotSource
HuaweipetalbotSource
Navernaver.me/spdSource
Yahoo!slurpSource
BaidubaiduspiderSource
Sogousogou web spider sogou wap spider sogou pic spiderSource
ExsaladeexabotSource
KokokcoccocbotSource
SeznanseznamBotSource
Googlegooglebot AdsBot-Google Storebot-Google APIs-Google AdsBot-Google-Mobile Mediapartners-Google Googlebot-Image Googlebot-News Googlebot-Video AdsBot-Google-Mobile-Apps FeedFetcher-Google Google-Read-Aloud Google-Site-VerificationSource
Yandexyandex.com/botsSource
Byte JumpingbytespiderSource
DuckduckgoduckduckbotSource
AhrefsAhrefsBot AhrefsSiteAuditSource
Facebookfacebookexternalhit facebookcatalogSource
PinterestPinterestbotSource
SincloroCincrawSource
Semrushhttp://www.semrush.com/bot.html
SekportSeekportBotSource
AmazonAmazonbot/0.1Source
Data DogdatadogSource