分享一些恶意爬虫的UA特征

使用宝塔面板的可以添加在Nginx防火墙里面的UA黑名单（自己整理，仅供参考）

paloaltonetworks

Go-http-client

webprosbot

python

github

libcurl

fasthttp

ZoominfoBot

Knowledge AI

RepoLookoutBot

seokicks

serpstatbot

SurdotlyBot

DataForSeoBot

DigExt

HttpClient

MJ12bot

AhrefsBot

DotBot

SemrushBot

MegaIndex

BLEXBot

ExtLinksBot

DnyzBot

Web-Crawler

Ezooms

Linguee Bot

WellKnownBot

MauiBot

HubSpot

Criteo

Barkrowler

Grapeshot

Yellowbrandprotectionbot

FlightDeckReports

Heritrix

AwarioBot