You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
|
|
|
|
# machete的hero爬虫脚本库
|
|
|
|
|
|
|
|
|
|
Hero scripts of machete.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
将支持以下平台基本数据抓取：
|
|
|
|
|
|
|
|
|
|
* 抖音网页版
|
|
|
|
|
* 快手网页版
|
|
|
|
|
* 西瓜视频网页版
|
|
|
|
|
* Bilibili
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
爬虫采集到的数据结构见Machete项目的TaJian皮肤文档。
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## 目录/文件说明
|
|
|
|
|
|
|
|
|
|
* bot - 针对各大平台的网页HTML解析类
|
|
|
|
|
* bypass - 针对各大平台的常用域名收集
|
|
|
|
|
* test - 类库测试代码
|
|
|
|
|
* tmp - 临时文件保存目录
|
|
|
|
|
* install_cloud.sh - hero服务端安装（非必需）
|
|
|
|
|
* install_hero.sh - hero客户端安装
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## 使用方法
|
|
|
|
|
|
|
|
|
|
下载本源码到本地后，执行下面命令安装依赖包：
|
|
|
|
|
```
|
|
|
|
|
npm install
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
如果你对npm和node不熟悉，请自行了解。
|
|
|
|
|
|
