Home » Author Archives: nyasin585

Scrapy – Pass argument to callback function

Sometime some of the data needs to be passed to the another callback function while using the scrapy framework. For that we can use the meta property of the scrapy request and pass our data there. In the callback function the data can be received using the response.meta.get(”) method. The sample entire scrapy code will…

Multithread python scraping script

While scraping multiple website links one at a time it would be time consuming. We can use the multithread features from the futures module of python to implement threading in python using the simplest way possible. The thread pool executor will going to run the number of threads we need to run at a time.

Login to linkedin using python

LinkedIn needs login to access information inside it. If you are into data mining and wanted to scrape linkedin data then selenium could be the best choice but it comes with the trade-off for time. Instead we can use python requests module to login to the linkedin and scrape the data. let’s import the requests…

Couldn’t get size error Ubuntu

When forcing a linux(ubuntu) there might be a file system issue and it after restarting it throws a dark terminal with error like ‘couldn’t get size 0x8000000’ So to solve the issue you have to use the ‘fsck’ command . In the error message it clearly shows the partition in which there is error ….

React Native ADB permission denied issue

While running the react native script for the first time there might be a permission issue of ADB. Instead of giving a full permission to ADB in the android_sdk/platform tools adb in linux it might happen. After giving the permission though it shows the permission issue while running the react native app in the mobile…

Scrape profile pictures from multiple facebook ID’s

Facebook graph API is excellent tool for scrape the facebook data. Unfortunately the token has 1 hour expiration time and no more than 400-500 request can be done with a single token. If you have around 50 k facebook ID and wanted to extract all the profile picture then there is one trick through which…

Getting started with sqlite react-native (part 1)

Sqlite is the sql database engine for the mobile devices. It provides the relational database features as in the SQL only the difference is the basic syntax format. Expo has recently added the support for the SQLite for RN (React Native). In this post I will simply share a insert operation only. First import the…

1 2