前几天参加一个面试,准备了几天的js逆向,二面坠机了😶🌫️ (我是fw) 把最近速成的js逆向总结一下然后继续玩渗透了
短期速成,质量不齐,仅供参考
现状
国内
对于前端加密通常会用到一些专门的反爬框架例如瑞数,在很多政企网站里面能够看到,然后一些大厂什么的会用到自研的一些vmp来做前端加密.vmp还有其他例如webpack打包的手段实现了对js代码的强混淆,让传统的手撕逆向难度飙升,此外还有常见例如滑块啊什么的操作.当前了解到的比较前沿的手法是补环境或者通过rpc,有一些验证码挑战的话可能还需要模拟浏览器操作.
国外
大部分都是用的反爬产品例如cloudflare、akamai等,跟瑞数差不多吧 🤨常见的比如说cloudflare的5秒盾,然后还有一些指纹检测,行为风控什么的,会用到一些开源项目梭哈,后面会讲.
VMP(Virtual Machine Protection)是一种代码保护技术,它把原本的逻辑编译成一套自定义“虚拟指令”,再由一个内置解释器去执行。也就是说代码不再按正常 JS 运行,而是在一个“虚拟机”里跑,从而隐藏真实逻辑,让人难以看懂、分析或还原。
补环境操作
主要是因为一些例如vmp的混淆难以调试和逆向,于是干脆直接将js代码移植到本地进行调用调教,然后插入js脚本进行自动化调用.不过本地的node.js或者python环境的js实现跟浏览器的不一样,缺少很多原型对象等导致加密函数难以正常使用或者跟浏览器生成的不一样.这个时候就需要补上环境让加密函数能够在本地实现跟浏览器一样的效果,高手的话都是纯手补,拉下来看缺啥补啥,也比较麻烦.此外也可以套用别人现成的环境或者专门的开源项目
纯手补
网上教程很多,时间充裕可以体验学习下,篇幅有限就不带着体验了(x🤡
其实是根本不会,不然也不叫速成了,咕咕嘎嘎~啊!

补环境相关的开源项目
现成通用的很多,可以自己找,这里说一个专门针对瑞数这个产品的
https://github.com/pysunday/sdenv
没实操,因为很多政企🥲,请务必在遵守网络安全法的情况下进行合法安全测试
JsRPC操作
https://github.com/jxhczhl/JsRpc
自己简单学一下大概操作,或者直接让AI 帮你理解,不难
就是通过websocket连接生成接口直接调用浏览器环境里面的函数,所以说你只需要找到加密签名的是哪些函数即可以及其发包组成方式,不需要专门去逆向怎么加密签名的算法
逆向web前端
准备
起一个服务端,我这边用的Windows,再起一个高仿的测试页面😉 测试一下手机号密码登录

随便登录一下F12抓一个网络包看看启动器堆栈,发现一坨被混淆了,不过可以通过直接搜的方式找到大概加密位置,其实就是这个函数做的,可以多复制点上下文问问AI

当然这只是加密函数,还有签名函数

__NS_hxfalcon GET参数

请求头里cookie这一块,还有些乱七八糟的请求头

怎么找,直接把对应值拿去搜索,找一下上下文追一下找到签名函数即可,懒人直接拿去问AI也行,其余的就不展示了,大概就是这样找,然后整理一下用到了哪些函数以及最终发包的组成方式

这里说一下,其实很多参数是固定的指纹,可以直接读取当前浏览器的值,具体固定到什么程度可以自己测试控一下.
模拟登录
前面打好的断点不要删,点一下右上角的忽略断点然后刷新一下

注入js环境,https://github.com/jxhczhl/JsRpc/blob/main/resouces/JsEnv_Dev.js
复制里面的js代码直接在控制台输入

然后进行连接,在控制台输入
var rpc = new Hlclient("ws://127.0.0.1:12080/ws?group=mytest&name=loginpage");
因为这些个加密函数都是临时eval加载的,所以说需要做一个动调再给他全部提升到全局,打开断点重新测试停在断点的时候在控制台输入
d2luZG93LlFuID0gUW47DQp3aW5kb3cuaWkgPSBpaTsNCndpbmRvdy5ZaSA9IFlpOw0Kd2luZG93Lm9pID0gb2k7DQp3aW5kb3cuSmkgPSBKaTsNCndpbmRvdy5RaSA9IFFpOw0Kd2luZG93LlduID0gV247DQp3aW5kb3cuSm4gPSBKbjsNCndpbmRvdy5kdCA9IGR0Ow==请自行解码,后面也是

并注册RPC方法,也是直接输入在控制台
cnBjLnJlZ0FjdGlvbigiZW5jcnlwdF9waG9uZV9sb2dpbiIsIGZ1bmN0aW9uIChyZXNvbHZlLCBwYXJhbSkgew0KICB0cnkgew0KICAgIGNvbnN0IHJldCA9IHdpbmRvdy5Rbih7DQogICAgICBwaG9uZTogcGFyYW0ucGhvbmUsDQogICAgICBwYXNzd29yZDogcGFyYW0ucGFzc3dvcmQsDQogICAgICBhY2NvdW50OiBwYXJhbS5hY2NvdW50IHx8ICIiLA0KICAgICAgZW1haWw6IHBhcmFtLmVtYWlsIHx8ICIiDQogICAgfSk7DQoNCiAgICByZXNvbHZlKEpTT04uc3RyaW5naWZ5KHJldCkpOw0KICB9IGNhdGNoIChlKSB7DQogICAgcmVzb2x2ZShKU09OLnN0cmluZ2lmeSh7DQogICAgICBvazogZmFsc2UsDQogICAgICBlcnJvcjogU3RyaW5nKGUpLA0KICAgICAgc3RhY2s6IGUgJiYgZS5zdGFjayA/IGUuc3RhY2sgOiAiIg0KICAgIH0pKTsNCiAgfQ0KfSk7DQoNCnJwYy5yZWdBY3Rpb24oInNpZ25fcGhvbmVfbG9naW4iLCBmdW5jdGlvbiAocmVzb2x2ZSwgcGFyYW0pIHsNCiAgUHJvbWlzZS5yZXNvbHZlKHdpbmRvdy5paShwYXJhbS51cmwsIHBhcmFtLmZvcm1EYXRhIHx8IHt9KSkNCiAgICAudGhlbihmdW5jdGlvbiAocmV0KSB7DQogICAgICBpZiAocmV0ICYmIHR5cGVvZiByZXQgPT09ICJvYmplY3QiICYmICJocmVmIiBpbiByZXQpIHsNCiAgICAgICAgcmVzb2x2ZShyZXQuaHJlZik7DQogICAgICB9IGVsc2Ugew0KICAgICAgICByZXNvbHZlKFN0cmluZyhyZXQpKTsNCiAgICAgIH0NCiAgICB9KQ0KICAgIC5jYXRjaChmdW5jdGlvbiAoZSkgew0KICAgICAgcmVzb2x2ZShKU09OLnN0cmluZ2lmeSh7DQogICAgICAgIG9rOiBmYWxzZSwNCiAgICAgICAgZXJyb3I6IFN0cmluZyhlKSwNCiAgICAgICAgc3RhY2s6IGUgJiYgZS5zdGFjayA/IGUuc3RhY2sgOiAiIg0KICAgICAgfSkpOw0KICAgIH0pOw0KfSk7DQoNCnJwYy5yZWdBY3Rpb24oImdldF9jb29raWUiLCBmdW5jdGlvbiAocmVzb2x2ZSwgcGFyYW0pIHsNCiAgdHJ5IHsNCiAgICByZXNvbHZlKGRvY3VtZW50LmNvb2tpZSB8fCAiIik7DQogIH0gY2F0Y2ggKGUpIHsNCiAgICByZXNvbHZlKEpTT04uc3RyaW5naWZ5KHsNCiAgICAgIG9rOiBmYWxzZSwNCiAgICAgIGVycm9yOiBTdHJpbmcoZSksDQogICAgICBzdGFjazogZSAmJiBlLnN0YWNrID8gZS5zdGFjayA6ICIiDQogICAgfSkpOw0KICB9DQp9KTsNCg0KcnBjLnJlZ0FjdGlvbigiZ2V0X2t3d19pbmZvIiwgZnVuY3Rpb24gKHJlc29sdmUsIHBhcmFtKSB7DQogIHRyeSB7DQogICAgdmFyIGZwS2V5ID0gImt3ZnYxIjsNCiAgICB2YXIgY29va2llVmFsdWUgPSBudWxsOw0KICAgIHZhciBsb2NhbFZhbHVlID0gbnVsbDsNCiAgICB2YXIga3dwc2VjVmFsdWUgPSBudWxsOw0KDQogICAgdHJ5IHsNCiAgICAgIGxvY2FsVmFsdWUgPSBsb2NhbFN0b3JhZ2UuZ2V0SXRlbShmcEtleSkgfHwgbnVsbDsNCiAgICB9IGNhdGNoIChlKSB7fQ0KDQogICAgdHJ5IHsNCiAgICAgIHZhciBtID0gZG9jdW1lbnQuY29va2llLm1hdGNoKG5ldyBSZWdFeHAoZnBLZXkgKyAiPShbXjtdKykiKSk7DQogICAgICBjb29raWVWYWx1ZSA9IG0gPyBkZWNvZGVVUklDb21wb25lbnQobVsxXSkgOiBudWxsOw0KICAgIH0gY2F0Y2ggKGUpIHt9DQoNCiAgICB0cnkgew0KICAgICAga3dwc2VjVmFsdWUgPSB3aW5kb3cua3dwc2VjICYmIHR5cGVvZiB3aW5kb3cua3dwc2VjLmdldERhdGEgPT09ICJmdW5jdGlvbiINCiAgICAgICAgPyB3aW5kb3cua3dwc2VjLmdldERhdGEoKQ0KICAgICAgICA6IG51bGw7DQogICAgfSBjYXRjaCAoZSkge30NCg0KICAgIHJlc29sdmUoSlNPTi5zdHJpbmdpZnkoew0KICAgICAgb2s6IHRydWUsDQogICAgICBmaW5nZXJwcmludEtleTogZnBLZXksDQogICAgICBjb29raWU6IGRvY3VtZW50LmNvb2tpZSB8fCAiIiwNCiAgICAgIGt3ZnYxX2Nvb2tpZTogY29va2llVmFsdWUsDQogICAgICBrd2Z2MV9sb2NhbFN0b3JhZ2U6IGxvY2FsVmFsdWUsDQogICAgICBrd3BzZWNfZGF0YToga3dwc2VjVmFsdWUNCiAgICB9KSk7DQogIH0gY2F0Y2ggKGUpIHsNCiAgICByZXNvbHZlKEpTT04uc3RyaW5naWZ5KHsNCiAgICAgIG9rOiBmYWxzZSwNCiAgICAgIGVycm9yOiBTdHJpbmcoZSksDQogICAgICBzdGFjazogZSAmJiBlLnN0YWNrID8gZS5zdGFjayA6ICIiDQogICAgfSkpOw0KICB9DQp9KTs=
最后忽略断点并放行,访问你的http://127.0.0.1:12080/ 应该可以看到注册好的方法,用python脚本访问接口进行登录测试
aW1wb3J0IGpzb24NCmltcG9ydCByZXF1ZXN0cw0KDQpKU1JQQ19CQVNFID0gImh0dHA6Ly8xMjcuMC4wLjE6MTIwODAvZ28iDQpHUk9VUCA9ICJteXRlc3QiDQoNCkJBU0VfVVJMID0gImh0dHBzOi8vaWQua3VhaXNob3UuY29tIg0KTE9HSU5fQVBJID0gQkFTRV9VUkwgKyAiL3Bhc3Mva3VhaXNob3UvbG9naW4vcGhvbmUvdjIiDQoNCg0KZGVmIGNhbGxfanNycGMoYWN0aW9uLCBwYXJhbT1Ob25lLCB0aW1lb3V0PTIwKToNCiAgICBkYXRhID0gew0KICAgICAgICAiZ3JvdXAiOiBHUk9VUCwNCiAgICAgICAgImFjdGlvbiI6IGFjdGlvbg0KICAgIH0NCiAgICBpZiBwYXJhbSBpcyBub3QgTm9uZToNCiAgICAgICAgZGF0YVsicGFyYW0iXSA9IGpzb24uZHVtcHMocGFyYW0sIGVuc3VyZV9hc2NpaT1GYWxzZSkNCg0KICAgIHIgPSByZXF1ZXN0cy5wb3N0KEpTUlBDX0JBU0UsIGRhdGE9ZGF0YSwgdGltZW91dD10aW1lb3V0KQ0KICAgIHIucmFpc2VfZm9yX3N0YXR1cygpDQoNCiAgICB0cnk6DQogICAgICAgIHJldHVybiBqc29uLmxvYWRzKHIudGV4dCkNCiAgICBleGNlcHQ6DQogICAgICAgIHJldHVybiByLnRleHQNCg0KDQojIC0tLS0tLS0tLS0g5Yqg5a+GIC0tLS0tLS0tLS0NCmRlZiBnZXRfZW5jcnlwdChwaG9uZSwgcGFzc3dvcmQpOg0KICAgIHJlcyA9IGNhbGxfanNycGMoImVuY3J5cHRfcGhvbmVfbG9naW4iLCB7DQogICAgICAgICJwaG9uZSI6IHBob25lLA0KICAgICAgICAicGFzc3dvcmQiOiBwYXNzd29yZCwNCiAgICAgICAgImFjY291bnQiOiAiIiwNCiAgICAgICAgImVtYWlsIjogIiINCiAgICB9KQ0KDQogICAgaW5uZXIgPSByZXMuZ2V0KCJkYXRhIikNCiAgICBpZiBpc2luc3RhbmNlKGlubmVyLCBzdHIpOg0KICAgICAgICBpbm5lciA9IGpzb24ubG9hZHMoaW5uZXIpDQoNCiAgICBpZiBpbm5lci5nZXQoIm9rIikgaXMgRmFsc2U6DQogICAgICAgIHJhaXNlIFJ1bnRpbWVFcnJvcihpbm5lcikNCg0KICAgIHJldHVybiBpbm5lcg0KDQoNCiMgLS0tLS0tLS0tLSDnrb7lkI0gLS0tLS0tLS0tLQ0KZGVmIGdldF9zaWduZWRfdXJsKHVybCwgZm9ybSk6DQogICAgcmVzID0gY2FsbF9qc3JwYygic2lnbl9waG9uZV9sb2dpbiIsIHsNCiAgICAgICAgInVybCI6IHVybCwNCiAgICAgICAgImZvcm1EYXRhIjogZm9ybQ0KICAgIH0pDQoNCiAgICBpZiBpc2luc3RhbmNlKHJlcywgZGljdCk6DQogICAgICAgIGlubmVyID0gcmVzLmdldCgiZGF0YSIsICIiKQ0KICAgICAgICBpZiBpc2luc3RhbmNlKGlubmVyLCBzdHIpOg0KICAgICAgICAgICAgcmV0dXJuIGlubmVyLnN0cmlwKCkNCiAgICAgICAgcmV0dXJuIHN0cihpbm5lcikuc3RyaXAoKQ0KDQogICAgaWYgaXNpbnN0YW5jZShyZXMsIHN0cik6DQogICAgICAgIHJldHVybiByZXMuc3RyaXAoKQ0KDQogICAgcmV0dXJuICIiDQoNCiMgLS0tLS0tLS0tLSBjb29raWUgLS0tLS0tLS0tLQ0KZGVmIGdldF9jb29raWUoKToNCiAgICByZXMgPSBjYWxsX2pzcnBjKCJnZXRfY29va2llIikNCg0KICAgIGlmIGlzaW5zdGFuY2UocmVzLCBkaWN0KToNCiAgICAgICAgaW5uZXIgPSByZXMuZ2V0KCJkYXRhIiwgIiIpDQogICAgICAgIGlmIGlzaW5zdGFuY2UoaW5uZXIsIHN0cik6DQogICAgICAgICAgICByZXR1cm4gaW5uZXIuc3RyaXAoKQ0KICAgICAgICByZXR1cm4gc3RyKGlubmVyKS5zdHJpcCgpDQoNCiAgICBpZiBpc2luc3RhbmNlKHJlcywgc3RyKToNCiAgICAgICAgcmV0dXJuIHJlcy5zdHJpcCgpDQoNCiAgICByZXR1cm4gIiINCg0KDQojIC0tLS0tLS0tLS0ga3d3IC0tLS0tLS0tLS0NCmRlZiBnZXRfa3d3X2luZm8oKToNCiAgICByZXMgPSBjYWxsX2pzcnBjKCJnZXRfa3d3X2luZm8iLCB7fSkNCg0KICAgIHByaW50KCJnZXRfa3d3X2luZm8gcmF3ID0+IiwgcmVwcihyZXMpKQ0KDQogICAgaWYgaXNpbnN0YW5jZShyZXMsIGRpY3QpOg0KICAgICAgICBpbm5lciA9IHJlcy5nZXQoImRhdGEiLCAiIikNCiAgICAgICAgaWYgaXNpbnN0YW5jZShpbm5lciwgc3RyKToNCiAgICAgICAgICAgIGlubmVyID0ganNvbi5sb2Fkcyhpbm5lcikNCiAgICAgICAgcmV0dXJuIGlubmVyIGlmIGlzaW5zdGFuY2UoaW5uZXIsIGRpY3QpIGVsc2Uge30NCg0KICAgIGlmIGlzaW5zdGFuY2UocmVzLCBzdHIpOg0KICAgICAgICBzID0gcmVzLnN0cmlwKCkNCiAgICAgICAgaWYgcy5zdGFydHN3aXRoKCJ7Iikgb3Igcy5zdGFydHN3aXRoKCJbIik6DQogICAgICAgICAgICByZXR1cm4ganNvbi5sb2FkcyhzKQ0KDQogICAgcmV0dXJuIHt9DQoNCmRlZiBnZXRfa3d3KCk6DQogICAgaW5mbyA9IGdldF9rd3dfaW5mbygpDQogICAgcHJpbnQoInBhcnNlZCBrd3cgaW5mbyA9PiIsIGpzb24uZHVtcHMoaW5mbywgZW5zdXJlX2FzY2lpPUZhbHNlLCBpbmRlbnQ9MikpDQoNCiAgICAjIOS8mOWFiOeUqCBrd3BzZWNfZGF0Ye+8jOWFtuasoSBjb29raWXvvIzlho3lhbbmrKEgbG9jYWxTdG9yYWdlDQogICAgcmV0dXJuICgNCiAgICAgICAgaW5mby5nZXQoImt3cHNlY19kYXRhIikNCiAgICAgICAgb3IgaW5mby5nZXQoImt3ZnYxX2Nvb2tpZSIpDQogICAgICAgIG9yIGluZm8uZ2V0KCJrd2Z2MV9sb2NhbFN0b3JhZ2UiKQ0KICAgICAgICBvciAiIg0KICAgICkNCg0KZGVmIG5vcm1hbGl6ZV9oZWFkZXJzKGhlYWRlcnM6IGRpY3QpIC0+IGRpY3Q6DQogICAgb3V0ID0ge30NCiAgICBmb3IgaywgdiBpbiBoZWFkZXJzLml0ZW1zKCk6DQogICAgICAgIGlmIHYgaXMgTm9uZToNCiAgICAgICAgICAgIGNvbnRpbnVlDQogICAgICAgIG91dFtzdHIoayldID0gc3RyKHYpDQogICAgcmV0dXJuIG91dA0KDQoNCiMgLS0tLS0tLS0tLSDkuLvmtYHnqIsgLS0tLS0tLS0tLQ0KZGVmIG1haW4oKToNCiAgICBwaG9uZSA9ICIxMzgwMDEzODAwMCINCiAgICBwYXNzd29yZCA9ICIxMjM0NTZxd2VRV0UiDQoNCiAgICBwcmludCgiPT09IDEuIOWKoOWvhiA9PT0iKQ0KICAgIGVuYyA9IGdldF9lbmNyeXB0KHBob25lLCBwYXNzd29yZCkNCiAgICBwcmludChqc29uLmR1bXBzKGVuYywgaW5kZW50PTIsIGVuc3VyZV9hc2NpaT1GYWxzZSkpDQoNCiAgICAjIOaehOmAoCBib2R577yI5b+F6aG75ZKM5rWP6KeI5Zmo5LiA6Ie077yJDQogICAgZm9ybSA9IHsNCiAgICAgICAgInNpZCI6ICJrdWFpc2hvdS53ZWIuYXBpIiwNCiAgICAgICAgInBhc3N3b3JkIjogZW5jWyJwYXNzd29yZCJdLA0KICAgICAgICAiaWdub3JlUHdkIjogZW5jLmdldCgiaWdub3JlUHdkIiwgRmFsc2UpLA0KICAgICAgICAicGhvbmUiOiBlbmNbInBob25lIl0sDQogICAgICAgICJpZ25vcmVBY2NvdW50IjogZW5jLmdldCgiaWdub3JlQWNjb3VudCIsIEZhbHNlKSwNCiAgICAgICAgImNvdW50cnlDb2RlIjogIis4NiIsDQogICAgICAgICJjaGFubmVsVHlwZSI6ICJQQ19QQUdFIiwNCiAgICAgICAgImlzV2ViU2lnNCI6IFRydWUsDQogICAgfQ0KDQogICAgcHJpbnQoIlxuPT09IDIuIOetvuWQjSA9PT0iKQ0KICAgIHNpZ25lZF91cmwgPSBnZXRfc2lnbmVkX3VybChMT0dJTl9BUEksIGZvcm0pDQogICAgcHJpbnQoc2lnbmVkX3VybCkNCg0KICAgIHByaW50KCJcbj09PSAzLiBjb29raWUgPT09IikNCiAgICBjb29raWUgPSBnZXRfY29va2llKCkNCiAgICBwcmludChjb29raWUpDQoNCiAgICBwcmludCgiXG49PT0gNC4ga3d3ID09PSIpDQogICAga3d3ID0gZ2V0X2t3dygpDQogICAgcHJpbnQoa3d3KQ0KDQogICAgIyBoZWFkZXJz77yI5YWz6ZSu77yJDQogICAgaGVhZGVycyA9IHsNCiAgICAgICAgIlVzZXItQWdlbnQiOiAiTW96aWxsYS81LjAiLA0KICAgICAgICAiQWNjZXB0IjogIiovKiIsDQogICAgICAgICJDb250ZW50LVR5cGUiOiAiYXBwbGljYXRpb24veC13d3ctZm9ybS11cmxlbmNvZGVkIiwNCiAgICAgICAgIk9yaWdpbiI6ICJodHRwczovL3Bhc3Nwb3J0Lmt1YWlzaG91LmNvbSIsDQogICAgICAgICJSZWZlcmVyIjogImh0dHBzOi8vcGFzc3BvcnQua3VhaXNob3UuY29tLyIsDQogICAgICAgICJDb29raWUiOiBjb29raWUsDQogICAgICAgICJrd3ciOiBrd3csDQogICAgfQ0KDQogICAgaGVhZGVycy51cGRhdGUoZW5jLmdldCgiZW5jcnlwdEhlYWRlcnMiLCB7fSkgb3Ige30pDQogICAgaGVhZGVycyA9IG5vcm1hbGl6ZV9oZWFkZXJzKGhlYWRlcnMpDQoNCiAgICBwcmludCgiXG49PT0gNS4g5pyA57uI6K+35rGCID09PSIpDQogICAgcHJpbnQoIlVSTDoiLCBzaWduZWRfdXJsKQ0KICAgIHByaW50KCJIRUFERVJTOiIsIGpzb24uZHVtcHMoaGVhZGVycywgaW5kZW50PTIsIGVuc3VyZV9hc2NpaT1GYWxzZSkpDQogICAgcHJpbnQoIkJPRFk6IiwgZm9ybSkNCg0KICAgIHByaW50KCI9PT0gaGVhZGVycyBjaGVjayA9PT0iKQ0KICAgIGZvciBrLCB2IGluIGhlYWRlcnMuaXRlbXMoKToNCiAgICAgICBwcmludChyZXByKGspLCB0eXBlKHYpLCByZXByKHYpKQ0KDQogICAgcHJpbnQoIlxuPT09IDYuIOWPkemAgeivt+axgiA9PT0iKQ0KICAgIHJlc3AgPSByZXF1ZXN0cy5wb3N0KA0KICAgICAgICBzaWduZWRfdXJsLA0KICAgICAgICBoZWFkZXJzPWhlYWRlcnMsDQogICAgICAgIGRhdGE9Zm9ybSwNCiAgICAgICAgdGltZW91dD0yMA0KICAgICkNCg0KICAgIHByaW50KCJcbj09PSA3LiDlk43lupQgPT09IikNCiAgICBwcmludChyZXNwLnN0YXR1c19jb2RlKQ0KICAgIHByaW50KHJlc3AudGV4dCkNCg0KDQppZiBfX25hbWVfXyA9PSAiX19tYWluX18iOg0KICAgIG1haW4oKQ==获取响应

不过这对吗?🙃这不对呀!🤪这是被风控了,过量的hook脚本测试还有断点调试触发了风控规则,这里建议刷新并删掉当前网站的所有cookie信息

正常的应该是紧接着触发滑块验证,验证成功后应该会给你另外的验证请求头(应该是这个😊)

关于滑块验证的话,你得先去看看这个滑块验证的实际逻辑是什么,通常会是你手动滑块触发js计算,然后返回一个正确值,有可能你仍然能直接通过类似的js函数调用解决这个问题,如果风控严格的话,比如说还要检验滑块滑动速率等参数是否拟人,这个时候就需要去做轨迹模拟,会用到可能贝塞尔曲线 、分段加速度模型 、添加随机抖动 等等手段进行模拟过风控.这就跟演示的jsrpc无关了🤪,也许这个工具就只能利用到这了
过CF5秒盾
这个应该很容易遇到,主打一个硬卡你几秒钟

简介一下,cloudflare通常会检测你当前的网络环境以及行为给一个评分,评分差的话会直接给你上一个5秒盾,然后还会检测发包的tls指纹甚至你点那个按钮的轨迹鼠标频率等等,一旦发现你不像人就会返回一个403,akamai也差不多,可能在指纹检测方面风控更严格,不过没有盾,另外的反爬产品都差不多
谢谢你,讲介使🤓
下面说一下一个组合拳加两个现成开源项目
组合拳
主要用到了以下三个github项目
https://github.com/ObjectAscended/CDP-bug-MouseEvent-.screenX-.screenY-patcher 绕过鼠标轨迹检测
https://github.com/g1879/DrissionPage/ 控制浏览器
https://github.com/lexiforest/curl_cffi 伪造tls指纹携带header快速访问cf站
主要思路是通过DrissionPage控制浏览器进行点击操作,直接的点击操作有弊端会被检测,然后第一个项目的浏览器插件用来绕过这个,最后拿到检测成功后的完整指纹凭证后用curl_cffi 模拟chrome124的tls指纹发出去就能够实现后续的高效率低消耗成本爬取,当然得控制速率还有代理等等,这里给一个demo仅供测试
pip install curl_cffi
pip install DrissionPage把上面那个浏览器插件下载下来把turnstilePatch文件夹在脚本同一目录下
aW1wb3J0IG9zDQppbXBvcnQgdGltZQ0KaW1wb3J0IGpzb24NCmltcG9ydCBzaHV0aWwNCmltcG9ydCB6aXBmaWxlDQppbXBvcnQgc3RyaW5nDQppbXBvcnQgdGVtcGZpbGUNCmltcG9ydCByYW5kb20NCmZyb20gdHlwaW5nIGltcG9ydCBMaXN0LCBPcHRpb25hbCwgVHVwbGUNCg0KZnJvbSBjdXJsX2NmZmkgaW1wb3J0IHJlcXVlc3RzIGFzIHNfcmVxdWVzdHMNCg0KZnJvbSBEcmlzc2lvblBhZ2UgaW1wb3J0IENocm9taXVtUGFnZSwgQ2hyb21pdW1PcHRpb25zDQoNCg0KVEVTVF9VUkwgPSAiaHR0cHM6Ly93d3cuY3ZlZGV0YWlscy5jb20iIA0KRkxBR19VUkwgPSAiaHR0cHM6Ly93d3cuY3ZlZGV0YWlscy5jb20vY3Zzcy1zY29yZS1jaGFydHMucGhwIiANCg0KVFVSTlNUSUxFX1BBVENIX1BBVEggPSBvcy5wYXRoLmFic3BhdGgoIi4vdHVybnN0aWxlUGF0Y2giKQ0KDQpVU0VfUFJPWFkgPSBGYWxzZQ0KUFJPWFlfQ09ORklHID0gew0KICAgICJob3N0IjogIjEyNy4wLjAuMSIsDQogICAgInBvcnQiOiA3ODk3LA0KICAgICJzY2hlbWUiOiAiaHR0cCIsDQp9DQoNCkhFQURMRVNTID0gRmFsc2UNCk1BWF9SRVRSWSA9IDE1ICAgIA0KUkVUUllfSU5URVJWQUwgPSAyDQoNCg0KZGVmIGV4dHJhY3RfemlwX2V4dGVuc2lvbih6aXBfcGF0aDogc3RyKSAtPiBzdHI6DQogICAgdGVtcF9kaXIgPSB0ZW1wZmlsZS5ta2R0ZW1wKHByZWZpeD0iZXh0XyIpDQogICAgd2l0aCB6aXBmaWxlLlppcEZpbGUoemlwX3BhdGgsICJyIikgYXMgemlwX3JlZjoNCiAgICAgICAgemlwX3JlZi5leHRyYWN0YWxsKHRlbXBfZGlyKQ0KICAgIHJldHVybiB0ZW1wX2Rpcg0KDQpkZWYgc2V0dXBfYnJvd3NlcihleHRlbnNpb25zOiBPcHRpb25hbFtMaXN0W3N0cl1dID0gTm9uZSwgcHJveHk6IE9wdGlvbmFsW2RpY3RdID0gTm9uZSwgaGVhZGxlc3M6IGJvb2wgPSBGYWxzZSkgLT4gVHVwbGVbQ2hyb21pdW1QYWdlLCBMaXN0W3N0cl1dOg0KICAgIGNvID0gQ2hyb21pdW1PcHRpb25zKCkNCiAgICB0ZW1wX3BhdGhzOiBMaXN0W3N0cl0gPSBbXQ0KICAgIGNvLmhlYWRsZXNzKGhlYWRsZXNzKQ0KDQogICAgIyAg5oyH5a6aIENocm9tZSDlkK/liqjlmajot6/lvoQNCiAgICBjby5zZXRfYnJvd3Nlcl9wYXRoKHIiRTpcXGNocm9tZTEyNFxcY2hyb21lLXdpbjY0XFxjaHJvbWUuZXhlIikNCg0KICAgIGFyZ3VtZW50cyA9IFsNCiAgICAgICAgIi1uby1maXJzdC1ydW4iLA0KICAgICAgICAiLWZvcmNlLWNvbG9yLXByb2ZpbGU9c3JnYiIsDQogICAgICAgICItbWV0cmljcy1yZWNvcmRpbmctb25seSIsDQogICAgICAgICItcGFzc3dvcmQtc3RvcmU9YmFzaWMiLA0KICAgICAgICAiLXVzZS1tb2NrLWtleWNoYWluIiwNCiAgICAgICAgIi1leHBvcnQtdGFnZ2VkLXBkZiIsDQogICAgICAgICItbm8tZGVmYXVsdC1icm93c2VyLWNoZWNrIiwNCiAgICAgICAgIi1kaXNhYmxlLWJhY2tncm91bmQtbW9kZSIsDQogICAgICAgICItZW5hYmxlLWZlYXR1cmVzPU5ldHdvcmtTZXJ2aWNlLE5ldHdvcmtTZXJ2aWNlSW5Qcm9jZXNzLExvYWRDcnlwdG9Ub2tlbkV4dGVuc2lvbixQZXJtdXRlVExTRXh0ZW5zaW9ucyIsDQogICAgICAgICItZGlzYWJsZS1mZWF0dXJlcz1GbGFzaERlcHJlY2F0aW9uV2FybmluZyxFbmFibGVQYXNzd29yZHNBY2NvdW50U3RvcmFnZSIsDQogICAgICAgICItZGVueS1wZXJtaXNzaW9uLXByb21wdHMiLA0KICAgICAgICAiLWFjY2VwdC1sYW5nPXpoLUNOIiwNCiAgICAgICAgIi0tbm8tc2FuZGJveCIsDQogICAgICAgICItLWd1ZXN0Ig0KICAgIF0NCiAgICBmb3IgYXJnIGluIGFyZ3VtZW50czoNCiAgICAgICAgY28uc2V0X2FyZ3VtZW50KGFyZykNCg0KICAgIHVzZXJfZGF0YV9kaXIgPSB0ZW1wZmlsZS5ta2R0ZW1wKHByZWZpeD0iY2hyb21lX3VzZXJfIikNCiAgICB0ZW1wX3BhdGhzLmFwcGVuZCh1c2VyX2RhdGFfZGlyKQ0KICAgIGNvLnNldF91c2VyX2RhdGFfcGF0aCh1c2VyX2RhdGFfZGlyKQ0KDQogICAgaWYgcHJveHk6DQogICAgICAgIGNvLnNldF9wcm94eShmIntwcm94eVsnc2NoZW1lJ119Oi8ve3Byb3h5Wydob3N0J119Ontwcm94eVsncG9ydCddfSIpDQoNCiAgICBpZiBleHRlbnNpb25zIGFuZCBvcy5wYXRoLmV4aXN0cyhleHRlbnNpb25zWzBdKToNCiAgICAgICAgZm9yIGV4dCBpbiBleHRlbnNpb25zOg0KICAgICAgICAgICAgcCA9IGV4dHJhY3RfemlwX2V4dGVuc2lvbihleHQpIGlmIGV4dC5sb3dlcigpLmVuZHN3aXRoKCIuemlwIikgZWxzZSBvcy5wYXRoLmFic3BhdGgoZXh0KQ0KICAgICAgICAgICAgaWYgZXh0Lmxvd2VyKCkuZW5kc3dpdGgoIi56aXAiKToNCiAgICAgICAgICAgICAgICB0ZW1wX3BhdGhzLmFwcGVuZChwKQ0KICAgICAgICAgICAgY28uYWRkX2V4dGVuc2lvbihwYXRoPXApDQoNCiAgICBwYWdlID0gQ2hyb21pdW1QYWdlKGNvKQ0KICAgIHJldHVybiBwYWdlLCB0ZW1wX3BhdGhzDQoNCmRlZiBjZl9jbGljayhwYWdlOiBDaHJvbWl1bVBhZ2UsIHRpbWVvdXQ6IGludCA9IDEwKSAtPiBib29sOg0KICAgIHRyeToNCiAgICAgICAgY2hhbGxlbmdlX3NvbHV0aW9uID0gcGFnZS5lbGUoIkBuYW1lPWNmLXR1cm5zdGlsZS1yZXNwb25zZSIsIHRpbWVvdXQ9dGltZW91dCkNCiAgICAgICAgaWYgbm90IGNoYWxsZW5nZV9zb2x1dGlvbjogcmV0dXJuIEZhbHNlDQogICAgICAgIGNoYWxsZW5nZV93cmFwcGVyID0gY2hhbGxlbmdlX3NvbHV0aW9uLnBhcmVudCgpDQogICAgICAgIGNoYWxsZW5nZV9pZnJhbWUgPSBjaGFsbGVuZ2Vfd3JhcHBlci5zaGFkb3dfcm9vdC5lbGUoInRhZzppZnJhbWUiKQ0KICAgICAgICBjaGFsbGVuZ2VfaWZyYW1lX2JvZHkgPSBjaGFsbGVuZ2VfaWZyYW1lLmVsZSgidGFnOmJvZHkiKS5zaGFkb3dfcm9vdA0KICAgICAgICBjaGFsbGVuZ2VfYnV0dG9uID0gY2hhbGxlbmdlX2lmcmFtZV9ib2R5LmVsZSgidGFnOmlucHV0IikNCiAgICAgICAgaWYgY2hhbGxlbmdlX2J1dHRvbjoNCiAgICAgICAgICAgIGNoYWxsZW5nZV9idXR0b24uY2xpY2soKQ0KICAgICAgICAgICAgcmV0dXJuIFRydWUNCiAgICAgICAgcmV0dXJuIEZhbHNlDQogICAgZXhjZXB0Og0KICAgICAgICByZXR1cm4gRmFsc2UNCg0KZGVmIGZldGNoX3dpdGhfY3VybF9jZmZpKHVybDogc3RyLCBicm93c2VyX3BhZ2U6IENocm9taXVtUGFnZSwgY2FwdHVyZWRfaGVhZGVyczogZGljdCwgcHJveHk6IE9wdGlvbmFsW2RpY3RdKToNCiAgICBjb29raWVzID0ge2NbJ25hbWUnXTogY1sndmFsdWUnXSBmb3IgYyBpbiBicm93c2VyX3BhZ2UuY29va2llcygpfQ0KICAgIA0KICAgIGNsZWFuX2hlYWRlcnMgPSB7fQ0KICAgIGZvciBrLCB2IGluIGNhcHR1cmVkX2hlYWRlcnMuaXRlbXMoKToNCiAgICAgICAgaWYgay5zdGFydHN3aXRoKCc6Jyk6DQogICAgICAgICAgICBjb250aW51ZQ0KICAgICAgICBpZiBrLmxvd2VyKCkgaW4gWydjb250ZW50LWxlbmd0aCcsICdjb29raWUnLCAnaG9zdCcsICdhY2NlcHQtZW5jb2RpbmcnLCAnY29ubmVjdGlvbiddOg0KICAgICAgICAgICAgY29udGludWUNCiAgICAgICAgY2xlYW5faGVhZGVyc1trXSA9IHYNCg0KICAgIHByb3hpZXMgPSBOb25lDQogICAgaWYgcHJveHk6DQogICAgICAgIHBfdXJsID0gZiJ7cHJveHlbJ3NjaGVtZSddfTovL3twcm94eVsnaG9zdCddfTp7cHJveHlbJ3BvcnQnXX0iDQogICAgICAgIHByb3hpZXMgPSB7Imh0dHAiOiBwX3VybCwgImh0dHBzIjogcF91cmx9DQoNCiAgICBzZXNzaW9uID0gc19yZXF1ZXN0cy5TZXNzaW9uKGltcGVyc29uYXRlPSJjaHJvbWUxMjQiKQ0KICAgIA0KICAgIHRyeToNCiAgICAgICAgcmVzcCA9IHNlc3Npb24uZ2V0KA0KICAgICAgICAgICAgdXJsLA0KICAgICAgICAgICAgaGVhZGVycz1jbGVhbl9oZWFkZXJzLA0KICAgICAgICAgICAgY29va2llcz1jb29raWVzLA0KICAgICAgICAgICAgcHJveGllcz1wcm94aWVzLA0KICAgICAgICAgICAgdGltZW91dD0yNSwNCiAgICAgICAgICAgIGFsbG93X3JlZGlyZWN0cz1UcnVlDQogICAgICAgICkNCiAgICAgICAgcmV0dXJuIHJlc3ANCiAgICBleGNlcHQgRXhjZXB0aW9uIGFzIGU6DQogICAgICAgIHByaW50KGYiW2N1cmxfY2ZmaV0g6K+35rGC5Y+R55Sf5byC5bi4OiB7ZX0iKQ0KICAgICAgICByZXR1cm4gTm9uZQ0KDQpkZWYgbWFpbigpOg0KICAgIGV4dHJhX2V4dGVuc2lvbnMgPSBbXQ0KICAgIGlmIG9zLnBhdGguZXhpc3RzKFRVUk5TVElMRV9QQVRDSF9QQVRIKToNCiAgICAgICAgZXh0cmFfZXh0ZW5zaW9ucy5hcHBlbmQoVFVSTlNUSUxFX1BBVENIX1BBVEgpDQoNCiAgICBwYWdlID0gTm9uZQ0KICAgIHRlbXBfcGF0aHM6IExpc3Rbc3RyXSA9IFtdDQoNCiAgICB0cnk6DQogICAgICAgIHByaW50KCJbbWFpbl0g5q2j5Zyo5Yid5aeL5YyW5rWP6KeI5Zmo546v5aKDLi4uIikNCiAgICAgICAgcGFnZSwgdGVtcF9wYXRocyA9IHNldHVwX2Jyb3dzZXIoDQogICAgICAgICAgICBleHRlbnNpb25zPWV4dHJhX2V4dGVuc2lvbnMsDQogICAgICAgICAgICBwcm94eT1QUk9YWV9DT05GSUcgaWYgVVNFX1BST1hZIGVsc2UgTm9uZSwNCiAgICAgICAgICAgIGhlYWRsZXNzPUhFQURMRVNTLA0KICAgICAgICApDQoNCiAgICAgICAgcGFnZS5saXN0ZW4uc3RhcnQodGFyZ2V0cz1URVNUX1VSTCkNCg0KICAgICAgICBwcmludChmIlttYWluXSDmraPlnKjorr/pl646IHtURVNUX1VSTH0iKQ0KICAgICAgICBwYWdlLmdldChURVNUX1VSTCkNCg0KICAgICAgICBjYXB0dXJlZF9oZWFkZXJzID0gTm9uZQ0KICAgICAgICBjbGVhcmFuY2VfZm91bmQgPSBGYWxzZQ0KDQogICAgICAgIGZvciBpIGluIHJhbmdlKE1BWF9SRVRSWSk6DQogICAgICAgICAgICBwcmludChmIltzb2x2ZV0g5qOA5p+l6L+b5bqmOiB7aSsxfS97TUFYX1JFVFJZfS4uLiIpDQoNCiAgICAgICAgICAgIGZvciBwYWNrZXQgaW4gcGFnZS5saXN0ZW4uc3RlcHModGltZW91dD0xKToNCiAgICAgICAgICAgICAgICBpZiAnc2VjLWNoLXVhLWZ1bGwtdmVyc2lvbicgaW4gc3RyKHBhY2tldC5yZXF1ZXN0LmhlYWRlcnMpLmxvd2VyKCk6DQogICAgICAgICAgICAgICAgICAgIGNhcHR1cmVkX2hlYWRlcnMgPSBkaWN0KHBhY2tldC5yZXF1ZXN0LmhlYWRlcnMpDQogICAgICAgICAgICAgICAgICAgIHByaW50KCJbK10g5a6e5pe25o2V6I6377ya5YWo6YeP54m55b6BIEhlYWRlcnMg5o+Q5Y+W5oiQ5YqfISIpDQoNCiAgICAgICAgICAgIGNvb2tpZXNfZGljdCA9IHtjWyduYW1lJ106IGNbJ3ZhbHVlJ10gZm9yIGMgaW4gcGFnZS5jb29raWVzKCl9DQogICAgICAgICAgICBpZiAiY2ZfY2xlYXJhbmNlIiBpbiBjb29raWVzX2RpY3QgYW5kICJKdXN0IGEgbW9tZW50IiBub3QgaW4gcGFnZS50aXRsZToNCiAgICAgICAgICAgICAgICBwcmludChmIlsrXSDpqozor4HpgJrov4fvvIHlt7LojrflvpcgY2ZfY2xlYXJhbmNlOiB7Y29va2llc19kaWN0WydjZl9jbGVhcmFuY2UnXVs6MTVdfS4uLiIpDQogICAgICAgICAgICAgICAgY2xlYXJhbmNlX2ZvdW5kID0gVHJ1ZQ0KICAgICAgICAgICAgICAgIGJyZWFrDQogICAgICAgICAgICANCiAgICAgICAgICAgIGNmX2NsaWNrKHBhZ2UsIHRpbWVvdXQ9MikNCiAgICAgICAgICAgIHRpbWUuc2xlZXAoUkVUUllfSU5URVJWQUwpDQoNCiAgICAgICAgaWYgY2xlYXJhbmNlX2ZvdW5kOg0KICAgICAgICAgICAgaWYgbm90IGNhcHR1cmVkX2hlYWRlcnM6DQogICAgICAgICAgICAgICAgcmVzID0gcGFnZS5saXN0ZW4ud2FpdCh0aW1lb3V0PTIpDQogICAgICAgICAgICAgICAgaWYgcmVzOiBjYXB0dXJlZF9oZWFkZXJzID0gZGljdChyZXMucmVxdWVzdC5oZWFkZXJzKQ0KDQogICAgICAgICAgICBwcmludCgiWypdIOato+WcqOWIh+aNouiHsyBjdXJsX2NmZmkg6I635Y+W5a6M5pW0IEhUTUwuLi4iKQ0KICAgICAgICAgICAgdGltZS5zbGVlcCgxKQ0KDQogICAgICAgICAgICByZXNwb25zZSA9IGZldGNoX3dpdGhfY3VybF9jZmZpKA0KICAgICAgICAgICAgICAgIEZMQUdfVVJMLCANCiAgICAgICAgICAgICAgICBwYWdlLCANCiAgICAgICAgICAgICAgICBjYXB0dXJlZF9oZWFkZXJzLCANCiAgICAgICAgICAgICAgICBQUk9YWV9DT05GSUcgaWYgVVNFX1BST1hZIGVsc2UgTm9uZQ0KICAgICAgICAgICAgKQ0KICAgICAgICAgICAgDQogICAgICAgICAgICBpZiByZXNwb25zZToNCiAgICAgICAgICAgICAgICBwcmludChmIlxuW+e7k+aenF0g54q25oCB56CBOiB7cmVzcG9uc2Uuc3RhdHVzX2NvZGV9IikNCiAgICAgICAgICAgICAgICBwcmludCgiPSIgKiA2MCkNCiAgICAgICAgICAgICAgICANCiAgICAgICAgICAgICAgICBmdWxsX2h0bWwgPSByZXNwb25zZS50ZXh0DQogICAgICAgICAgICAgICAgIyDlj6/ku6Xnm7TmjqXovpPlh7rmlbTkuKpIVE1M77yM6ZqP5L6/DQogICAgICAgICAgICAgICAgIyBwcmludChmdWxsX2h0bWwpDQogICAgICAgICAgICAgICAgDQogICAgICAgICAgICAgICAgcHJpbnQoIj0iICogNjApDQogICAgICAgICAgICAgICAgDQogICAgICAgICAgICAgICAgIyDkv53lrZjliLDmnKzlnLDmlofku7YNCiAgICAgICAgICAgICAgICB3aXRoIG9wZW4oInJlc3VsdC5odG1sIiwgInciLCBlbmNvZGluZz0idXRmLTgiKSBhcyBmOg0KICAgICAgICAgICAgICAgICAgICBmLndyaXRlKGZ1bGxfaHRtbCkNCiAgICAgICAgICAgICAgICBwcmludChmIlvmj5DnpLpdIOWujOaVtOWGheWuueW3suWQjOatpeS/neWtmOiHszoge29zLnBhdGguYWJzcGF0aCgncmVzdWx0Lmh0bWwnKX0iKQ0KICAgICAgICAgICAgICAgIA0KICAgICAgICAgICAgICAgIGlmIHJlc3BvbnNlLnN0YXR1c19jb2RlICE9IDIwMDoNCiAgICAgICAgICAgICAgICAgICAgcHJpbnQoZiJcblvorablkYpdIOeKtuaAgeeggemdnjIwMO+8jOWPr+iDveinpuWPkeS6huS6jOe6p+agoemqjOaIliBIZWFkZXIg5Lyg6YCS5pyJ6K+vIikNCiAgICAgICAgICAgIGVsc2U6DQogICAgICAgICAgICAgICAgcHJpbnQoIlstXSBjdXJsX2NmZmkg6K+35rGC5pyq6IO95oiQ5Yqf6L+U5ZueIikNCiAgICAgICAgZWxzZToNCiAgICAgICAgICAgIHByaW50KCJbLV0g5pyq6IO96YCa6L+HIENsb3VkZmxhcmUg6aqM6K+B77yM6K+35qOA5p+l5Luj55CGIElQIOaYr+WQpueos+WumiIpDQoNCiAgICAgICAgaW5wdXQoIlxuW+WujOaIkF0g5oyJ5Zue6L2m6ZSu6YCA5Ye65bm25riF55CG5Li05pe255uu5b2VLi4uIikNCg0KICAgIGZpbmFsbHk6DQogICAgICAgIGlmIHBhZ2U6DQogICAgICAgICAgICBwYWdlLnF1aXQoKQ0KICAgICAgICBmb3IgcCBpbiB0ZW1wX3BhdGhzOg0KICAgICAgICAgICAgaWYgb3MucGF0aC5pc2RpcihwKToNCiAgICAgICAgICAgICAgICB0cnk6DQogICAgICAgICAgICAgICAgICAgIHNodXRpbC5ybXRyZWUocCwgaWdub3JlX2Vycm9ycz1UcnVlKQ0KICAgICAgICAgICAgICAgIGV4Y2VwdDoNCiAgICAgICAgICAgICAgICAgICAgcGFzcw0KDQppZiBfX25hbWVfXyA9PSAiX19tYWluX18iOg0KICAgIG1haW4oKQ==注意改一下代理换成自己的端口,然后初步给的是两个url,一个初始url用于获取指纹信息凭证,另一个是指定需要爬取的页面,然后下载一个chrome124作为浏览器,最后运行demo会打开指定的chrome124然后等待cf按钮加载并点击,最后获取到完整的指纹凭证后用curl_cffi进行爬虫指定页面,并生成一个文件放在同一目录

FlareSolverr
这里简单就用docker进行测试
docker run -d \
--name=flaresolverr \
-p 8191:8191 \
-e LOG_LEVEL=debug \
-e TZ=Asia/Shanghai \
-e NO_PROXY=localhost,127.0.0.1,::1 \
-e no_proxy=localhost,127.0.0.1,::1 \
-e HTTP_PROXY= \
-e HTTPS_PROXY= \
-e ALL_PROXY= \
--restart unless-stopped \
ghcr.io/flaresolverr/flaresolverr:latest然后直接进行curl接口调用
curl -X POST http://127.0.0.1:8191/v1 \
-H "Content-Type: application/json" \
-d '{
"cmd":"request.get",
"url":"https://www.cvedetails.com/",
"proxy":{
"url":"http://172.17.0.1:7897"
}
}'
全程通过浏览器进行,比较慢,按原理来说应该不如我那个组合拳,然后依赖实时更新,如果cf更新策略这个项目没及时更新就不行了
里面关于proxy部分用的ip是kali在docker网卡中的
Scrapling
这个就比较牛,快然后其他产品akamai什么的也能适配,可以写成skill让AI智能体用这个进行爬虫什么的,很强
依旧拉docker进行测试
docker pull pyd4vinci/scrapling:latest可以让AI给你写一个接口进行交互
aW1wb3J0IGpzb24KaW1wb3J0IG9zCmltcG9ydCByZQppbXBvcnQgc3lzCmZyb20gZGF0ZXRpbWUgaW1wb3J0IGRhdGV0aW1lLCBVVEMKCmZyb20gc2NyYXBsaW5nLmZldGNoZXJzIGltcG9ydCBGZXRjaGVyCgoKZGVmIGxvZyhtc2c6IHN0cikgLT4gTm9uZToKICAgIHByaW50KGYiWytdIHttc2d9IikKCgpkZWYgd2Fybihtc2c6IHN0cikgLT4gTm9uZToKICAgIHByaW50KGYiWyFdIHttc2d9IikKCgpkZWYgZXJyKG1zZzogc3RyKSAtPiBOb25lOgogICAgcHJpbnQoZiJbLV0ge21zZ30iKQoKCmRlZiBlbnN1cmVfZGlyKHBhdGg6IHN0cikgLT4gTm9uZToKICAgIG9zLm1ha2VkaXJzKHBhdGgsIGV4aXN0X29rPVRydWUpCgoKZGVmIHNhZmVfZmlsZW5hbWUobmFtZTogc3RyKSAtPiBzdHI6CiAgICByZXR1cm4gcmUuc3ViKHIiW15cd1wtLl0rIiwgIl8iLCBuYW1lKVs6MTIwXQoKCmRlZiBib2R5X2J5dGVzKHBhZ2UpIC0+IGJ5dGVzOgogICAgZGF0YSA9IGdldGF0dHIocGFnZSwgImJvZHkiLCBiIiIpCiAgICBpZiBpc2luc3RhbmNlKGRhdGEsIGJ5dGVzKToKICAgICAgICByZXR1cm4gZGF0YQogICAgaWYgaXNpbnN0YW5jZShkYXRhLCBzdHIpOgogICAgICAgIHJldHVybiBkYXRhLmVuY29kZSgidXRmLTgiLCBlcnJvcnM9Imlnbm9yZSIpCiAgICByZXR1cm4gYiIiCgoKZGVmIGJvZHlfdGV4dChwYWdlKSAtPiBzdHI6CiAgICBkYXRhID0gYm9keV9ieXRlcyhwYWdlKQogICAgcmV0dXJuIGRhdGEuZGVjb2RlKCJ1dGYtOCIsIGVycm9ycz0iaWdub3JlIikuc3RyaXAoKQoKCmRlZiBleHRyYWN0X2lwKHRleHQ6IHN0cikgLT4gc3RyOgogICAgbSA9IHJlLnNlYXJjaChyIihbMC05XXsxLDN9KD86XC5bMC05XXsxLDN9KXszfSkiLCB0ZXh0KQogICAgaWYgbToKICAgICAgICByZXR1cm4gbS5ncm91cCgxKQogICAgbSA9IHJlLnNlYXJjaChyIihbMC05YS1mQS1GOl17Mix9KSIsIHRleHQpCiAgICBpZiBtOgogICAgICAgIHJldHVybiBtLmdyb3VwKDEpCiAgICByZXR1cm4gIiIKCgpkZWYgZmV0Y2goZmV0Y2hlcjogRmV0Y2hlciwgdXJsOiBzdHIsIHRpbWVvdXQ6IGludCA9IDQwKToKICAgIHJldHVybiBmZXRjaGVyLmdldCh1cmwsIHRpbWVvdXQ9dGltZW91dCkKCgpkZWYgY2hlY2tfaXAoZmV0Y2hlcjogRmV0Y2hlciwgbmFtZTogc3RyLCB1cmw6IHN0cikgLT4gZGljdDoKICAgIHRyeToKICAgICAgICBwYWdlID0gZmV0Y2goZmV0Y2hlciwgdXJsLCB0aW1lb3V0PTIwKQogICAgICAgIHRleHQgPSBib2R5X3RleHQocGFnZSkKICAgICAgICBpcCA9IGV4dHJhY3RfaXAodGV4dCkKICAgICAgICBsb2coZiJ7bmFtZX0gLT4gc3RhdHVzPXtwYWdlLnN0YXR1c30sIGJvZHk9e3RleHRbOjEyMF19IikKICAgICAgICByZXR1cm4gewogICAgICAgICAgICAibmFtZSI6IG5hbWUsCiAgICAgICAgICAgICJ1cmwiOiB1cmwsCiAgICAgICAgICAgICJzdGF0dXMiOiBwYWdlLnN0YXR1cywKICAgICAgICAgICAgImJvZHkiOiB0ZXh0LAogICAgICAgICAgICAiaXAiOiBpcCwKICAgICAgICAgICAgIm9rIjogcGFnZS5zdGF0dXMgPT0gMjAwLAogICAgICAgIH0KICAgIGV4Y2VwdCBFeGNlcHRpb24gYXMgZToKICAgICAgICBlcnIoZiJ7bmFtZX0gZmFpbGVkOiB7ZX0iKQogICAgICAgIHJldHVybiB7CiAgICAgICAgICAgICJuYW1lIjogbmFtZSwKICAgICAgICAgICAgInVybCI6IHVybCwKICAgICAgICAgICAgInN0YXR1cyI6IE5vbmUsCiAgICAgICAgICAgICJib2R5IjogIiIsCiAgICAgICAgICAgICJpcCI6ICIiLAogICAgICAgICAgICAib2siOiBGYWxzZSwKICAgICAgICAgICAgImVycm9yIjogc3RyKGUpLAogICAgICAgIH0KCgpkZWYgbWFpbigpOgogICAgaWYgbGVuKHN5cy5hcmd2KSA8IDI6CiAgICAgICAgcHJpbnQoIlVzYWdlOiBweXRob24gY2YucHkgPHVybD4iKQogICAgICAgIHN5cy5leGl0KDEpCgogICAgdGFyZ2V0X3VybCA9IHN5cy5hcmd2WzFdCiAgICBvdXRfZGlyID0gIm91dHB1dCIKICAgIGVuc3VyZV9kaXIob3V0X2RpcikKCiAgICBmZXRjaGVyID0gRmV0Y2hlcigpCgogICAgbG9nKGYiVGFyZ2V0IFVSTDoge3RhcmdldF91cmx9IikKCiAgICBwcm94eV9lbnYgPSB7CiAgICAgICAgIkhUVFBfUFJPWFkiOiBvcy5lbnZpcm9uLmdldCgiSFRUUF9QUk9YWSIsICIiKSwKICAgICAgICAiSFRUUFNfUFJPWFkiOiBvcy5lbnZpcm9uLmdldCgiSFRUUFNfUFJPWFkiLCAiIiksCiAgICAgICAgIkFMTF9QUk9YWSI6IG9zLmVudmlyb24uZ2V0KCJBTExfUFJPWFkiLCAiIiksCiAgICAgICAgImh0dHBfcHJveHkiOiBvcy5lbnZpcm9uLmdldCgiaHR0cF9wcm94eSIsICIiKSwKICAgICAgICAiaHR0cHNfcHJveHkiOiBvcy5lbnZpcm9uLmdldCgiaHR0cHNfcHJveHkiLCAiIiksCiAgICAgICAgImFsbF9wcm94eSI6IG9zLmVudmlyb24uZ2V0KCJhbGxfcHJveHkiLCAiIiksCiAgICB9CgogICAgcHJveHlfY29uZmlndXJlZCA9IGFueSh2LnN0cmlwKCkgZm9yIHYgaW4gcHJveHlfZW52LnZhbHVlcygpKQogICAgaWYgcHJveHlfY29uZmlndXJlZDoKICAgICAgICBsb2coIlByb3h5IGVudmlyb25tZW50IGRldGVjdGVkLiIpCiAgICBlbHNlOgogICAgICAgIHdhcm4oIk5vIHByb3h5IGVudmlyb25tZW50IHZhcmlhYmxlcyBkZXRlY3RlZC4iKQoKICAgIGlwX2NoZWNrcyA9IFsKICAgICAgICBjaGVja19pcChmZXRjaGVyLCAiaXBpZnkiLCAiaHR0cHM6Ly9hcGkuaXBpZnkub3JnIiksCiAgICAgICAgY2hlY2tfaXAoZmV0Y2hlciwgImlmY29uZmlnX21lIiwgImh0dHBzOi8vaWZjb25maWcubWUvaXAiKSwKICAgICAgICBjaGVja19pcChmZXRjaGVyLCAiaWNhbmhhemlwIiwgImh0dHBzOi8vaWNhbmhhemlwLmNvbSIpLAogICAgXQoKICAgIGlwcyA9IHNvcnRlZChzZXQoeFsiaXAiXSBmb3IgeCBpbiBpcF9jaGVja3MgaWYgeC5nZXQoImlwIikpKQogICAgaWYgaXBzOgogICAgICAgIGxvZyhmIk9ic2VydmVkIGVncmVzcyBJUChzKTogeycsICcuam9pbihpcHMpfSIpCiAgICAgICAgbG9nKCJQcm94eSBhcHBlYXJzIHRvIGJlIHdvcmtpbmcgb3IgYXQgbGVhc3Qgb3V0Ym91bmQgYWNjZXNzIGlzIG5vcm1hbC4iKQogICAgZWxzZToKICAgICAgICB3YXJuKCJDb3VsZCBub3QgZGV0ZXJtaW5lIGVncmVzcyBJUC4iKQoKICAgIHBhZ2VfcmVzdWx0ID0gewogICAgICAgICJ0YXJnZXRfdXJsIjogdGFyZ2V0X3VybCwKICAgICAgICAic3RhdHVzIjogTm9uZSwKICAgICAgICAiZmluYWxfdXJsIjogIiIsCiAgICAgICAgInRpdGxlIjogIiIsCiAgICAgICAgImh0bWxfZmlsZSI6ICIiLAogICAgICAgICJ0ZXh0X2ZpbGUiOiAiIiwKICAgICAgICAiZXJyb3IiOiAiIiwKICAgIH0KCiAgICB0cnk6CiAgICAgICAgcGFnZSA9IGZldGNoKGZldGNoZXIsIHRhcmdldF91cmwsIHRpbWVvdXQ9NDApCgogICAgICAgIHBhZ2VfcmVzdWx0WyJzdGF0dXMiXSA9IHBhZ2Uuc3RhdHVzCiAgICAgICAgcGFnZV9yZXN1bHRbImZpbmFsX3VybCJdID0gc3RyKHBhZ2UudXJsKQoKICAgICAgICB0aXRsZSA9ICIiCiAgICAgICAgdHJ5OgogICAgICAgICAgICBub2RlID0gcGFnZS5jc3NfZmlyc3QoInRpdGxlIikKICAgICAgICAgICAgaWYgbm9kZToKICAgICAgICAgICAgICAgIHRpdGxlID0gbm9kZS50ZXh0LnN0cmlwKCkKICAgICAgICBleGNlcHQgRXhjZXB0aW9uOgogICAgICAgICAgICBwYXNzCiAgICAgICAgcGFnZV9yZXN1bHRbInRpdGxlIl0gPSB0aXRsZQoKICAgICAgICBzdGFtcCA9IGRhdGV0aW1lLm5vdyhVVEMpLnN0cmZ0aW1lKCIlWSVtJWRfJUglTSVTIikKICAgICAgICBiYXNlID0gc2FmZV9maWxlbmFtZShmInBhZ2Vfe3N0YW1wfSIpCgogICAgICAgIHJhdyA9IGJvZHlfYnl0ZXMocGFnZSkKICAgICAgICB0eHQgPSBib2R5X3RleHQocGFnZSkKCiAgICAgICAgaHRtbF9wYXRoID0gb3MucGF0aC5qb2luKG91dF9kaXIsIGJhc2UgKyAiLmh0bWwiKQogICAgICAgIHR4dF9wYXRoID0gb3MucGF0aC5qb2luKG91dF9kaXIsIGJhc2UgKyAiLnR4dCIpCgogICAgICAgIHdpdGggb3BlbihodG1sX3BhdGgsICJ3YiIpIGFzIGY6CiAgICAgICAgICAgIGYud3JpdGUocmF3KQoKICAgICAgICB3aXRoIG9wZW4odHh0X3BhdGgsICJ3IiwgZW5jb2Rpbmc9InV0Zi04IikgYXMgZjoKICAgICAgICAgICAgZi53cml0ZSh0eHQpCgogICAgICAgIHBhZ2VfcmVzdWx0WyJodG1sX2ZpbGUiXSA9IGh0bWxfcGF0aAogICAgICAgIHBhZ2VfcmVzdWx0WyJ0ZXh0X2ZpbGUiXSA9IHR4dF9wYXRoCgogICAgICAgIGxvZyhmIkZldGNoIHN1Y2Nlc3M6IHN0YXR1cz17cGFnZS5zdGF0dXN9IikKICAgICAgICBsb2coZiJGaW5hbCBVUkw6IHtwYWdlLnVybH0iKQogICAgICAgIGxvZyhmIlRpdGxlOiB7dGl0bGV9IikKICAgICAgICBsb2coZiJTYXZlZCBIVE1MIHRvOiB7aHRtbF9wYXRofSIpCiAgICAgICAgbG9nKGYiU2F2ZWQgdGV4dCB0bzoge3R4dF9wYXRofSIpCgogICAgZXhjZXB0IEV4Y2VwdGlvbiBhcyBlOgogICAgICAgIHBhZ2VfcmVzdWx0WyJlcnJvciJdID0gc3RyKGUpCiAgICAgICAgZXJyKGYiRmV0Y2ggdGFyZ2V0IGZhaWxlZDoge2V9IikKCiAgICByZXN1bHQgPSB7CiAgICAgICAgInRpbWVfdXRjIjogZGF0ZXRpbWUubm93KFVUQykuaXNvZm9ybWF0KCksCiAgICAgICAgInByb3h5X2VudiI6IHByb3h5X2VudiwKICAgICAgICAiZWdyZXNzX2lwcyI6IGlwcywKICAgICAgICAiaXBfY2hlY2tzIjogaXBfY2hlY2tzLAogICAgICAgICJwYWdlX3Jlc3VsdCI6IHBhZ2VfcmVzdWx0LAogICAgfQoKICAgIHJlc3VsdF9wYXRoID0gb3MucGF0aC5qb2luKG91dF9kaXIsICJyZXN1bHQuanNvbiIpCiAgICB3aXRoIG9wZW4ocmVzdWx0X3BhdGgsICJ3IiwgZW5jb2Rpbmc9InV0Zi04IikgYXMgZjoKICAgICAgICBqc29uLmR1bXAocmVzdWx0LCBmLCBlbnN1cmVfYXNjaWk9RmFsc2UsIGluZGVudD0yKQoKICAgIGxvZyhmIlNhdmVkIHJlc3VsdCBKU09OIHRvOiB7cmVzdWx0X3BhdGh9IikKCgppZiBfX25hbWVfXyA9PSAiX19tYWluX18iOgogICAgbWFpbigpCg==然后直接进行调用即可,实际可能需要改一下目录挂载以及代理端口什么的
docker run --rm -it \
--network host \
-e ALL_PROXY=socks5://172.17.0.1:7897 \
-e all_proxy=socks5://172.17.0.1:7897 \
-v /home/kali/Desktop/jsre:/work \
-w /work \
--entrypoint /app/.venv/bin/python \
pyd4vinci/scrapling:latest \
/work/cf.py https://www.cvedetails.com/
后续实测发现很多其他网站都能爬,除了一些特殊的自定义网站,例如reddit.com
AI-js逆向审计平台
期间发现了一个AI js逆向平台的github项目,可以用api调用大模型帮你自动分析,很强
https://github.com/Valerian7/AI_JS_DEBUGGER
总结一下
有一个面试跟逆向爬虫有关,于是花了几天去速成精进这方面,都是各种找资源学,不懂就问AI,效率还是可以的.一面还行,二面坠机了,没准备好,况且自己也没啥实力能完美match这个岗位.🥲
感慨一下,AI这个小东西真强吧,我还看到很多用mcp手撕vmp的
What can i say!🚁
然后AI也能帮助很好去理解很多东西,反正对我而言,我能在极短时间内学到这些主要依靠AI帮助理解以及思路的转化(虽然可能看起来没啥难度)
未来我感觉开发迟早完蛋一大半,让我自己手搓上面的demo估计都得花好几天慢慢调,AI效率太快了而且拉低了很多东西的学习门槛,然后也导致可能更内卷吧🤔.学习门槛的降低不代表你能花更少时间学更多东西然后拿到更多薪资待遇,反而是花更多时间学更多东西才能跟得上通胀,不过学总比不学好吧
要么坐以待毙,或者狂奔力竭
叠个盾,上面这些东西基本上都是短时间速成,整个文章由 哈基米-AI生成🤪,可能会有很多瑕疵,所以仅作参考,加上各种技术时时俱进,上述测试时效性不强,以自己实践为准,另外转载务必标注,尊重原创
笔者常在991445030QQ群活跃,里面有很多佬可以一起交流技术