采集app天天快报相关资讯

采集app天天快报相关资讯

资源稀缺性，是很重要的；特别是app上资源，PC端很难找到。下面介绍你，怎么样采集app天天快报上的资讯。

app上的资源，PC端看不到。只能用抓包工具，来抓包数据源。

方法：安装安卓虚拟机，打开天天快报app，利用fiddler抓包。

抓取URL：以搜索“音乐”为例。

http://r.cnews.qq.com/search?uid=1034048478148815&store=274&hw=iPhone_7plus&devid=868857451279719&appversion=2.9.0&screen_width=576&mac=08%3A00%3A27%3Ad1%3A36%3A72&appver=17_areading_2.9.0&android_id=1034048478148815&origin_imei=868857451279719&sceneid=&mid=d39d21efb40861ec456ba62ac098030560e11fad&apptype=android&screen_height=1024&qqnetwork=wifi&omgbizid=73cf38b05c424f459dbaa6fd6fcbb5bd288d008021221b&Cookie=%20lskey%3D%3B%20luin%3D%3B%20skey%3D%3B%20uin%3D%3B%20logintype%3D0%3B&qn-rid=7dd2718a-a6b5-40be-ae94-8e23e0836d21&imsi_history=460006443421183&query=音乐&qn-sig=5bfecb6e5758e1f6b3ff9acb85da842a&activefrom=icon&imsi=460006443421183&omgid=cf268fc07aedcc46c33a7b078a18a2b9cd36001021221b

liunix环境下输入命令：

# curl -s "http://r.cnews.qq.com/search?uid=1034048478148815&store=274&hw=iPhone_7plus&devid=868857451279719&appversion=2.9.0&screen_width=576&mac=08%3A00%3A27%3Ad1%3A36%3A72&appver=17_areading_2.9.0&android_id=1034048478148815&origin_imei=868857451279719&sceneid=&mid=d39d21efb40861ec456ba62ac098030560e11fad&apptype=android&screen_height=1024&qqnetwork=wifi&omgbizid=73cf38b05c424f459dbaa6fd6fcbb5bd288d008021221b&Cookie=%20lskey%3D%3B%20luin%3D%3B%20skey%3D%3B%20uin%3D%3B%20logintype%3D0%3B&qn-rid=7dd2718a-a6b5-40be-ae94-8e23e0836d21&imsi_history=460006443421183&query=音乐&qn-sig=5bfecb6e5758e1f6b3ff9acb85da842a&activefrom=icon&imsi=460006443421183&omgid=cf268fc07aedcc46c33a7b078a18a2b9cd36001021221b"|jq .

// 大家可以看到数据源了，我们只需要URL。

采集URL，输入命令：

#

curl -s "http://r.cnews.qq.com/search?uid=1034048478148815&store=274&hw=iPhone_7plus&devid=868857451279719&appversion=2.9.0&screen_width=576&mac=08%3A00%3A27%3Ad1%3A36%3A72&appver=17_areading_2.9.0&android_id=1034048478148815&origin_imei=868857451279719&sceneid=&mid=d39d21efb40861ec456ba62ac098030560e11fad&apptype=android&screen_height=1024&qqnetwork=wifi&omgbizid=73cf38b05c424f459dbaa6fd6fcbb5bd288d008021221b&Cookie=%20lskey%3D%3B%20luin%3D%3B%20skey%3D%3B%20uin%3D%3B%20logintype%3D0%3B&qn-rid=7dd2718a-a6b5-40be-ae94-8e23e0836d21&imsi_history=460006443421183&query=音乐&qn-sig=5bfecb6e5758e1f6b3ff9acb85da842a&activefrom=icon&imsi=460006443421183&omgid=cf268fc07aedcc46c33a7b078a18a2b9cd36001021221b"|jq '.'|grep "\"url\":"

稍微处理下数据，得到纯净的URL。输入命令：

curl -s "http://r.cnews.qq.com/search?uid=1034048478148815&store=274&hw=iPhone_7plus&devid=868857451279719&appversion=2.9.0&screen_width=576&mac=08%3A00%3A27%3Ad1%3A36%3A72&appver=17_areading_2.9.0&android_id=1034048478148815&origin_imei=868857451279719&sceneid=&mid=d39d21efb40861ec456ba62ac098030560e11fad&apptype=android&screen_height=1024&qqnetwork=wifi&omgbizid=73cf38b05c424f459dbaa6fd6fcbb5bd288d008021221b&Cookie=%20lskey%3D%3B%20luin%3D%3B%20skey%3D%3B%20uin%3D%3B%20logintype%3D0%3B&qn-rid=7dd2718a-a6b5-40be-ae94-8e23e0836d21&imsi_history=460006443421183&query=音乐&qn-sig=5bfecb6e5758e1f6b3ff9acb85da842a&activefrom=icon&imsi=460006443421183&omgid=cf268fc07aedcc46c33a7b078a18a2b9cd36001021221b"|jq '.'|grep "\"url\":"|awk -F "\"" '{print $4}'

最后，得到URL，那么采集就比较简单了。可以利用火车头采集器或python脚本都可以。有什么问题可以咨询。