Sometime some of the data needs to be passed to the another callback function while using the scrapy framework. For that we can use the meta property of the scrapy request and pass our data there.
|
1 2 3 |
yield scrapy.Request('https://nextlink',callback=self.anothercallback,meta={ 'data':'mydata' }) |
In the callback function the data can be received using the response.meta.get(”) method.
|
1 2 3 |
def anothercallback(self,response): mydata = response.meta.get('mydata') |
The sample entire scrapy code will be like this
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
# -*- coding: utf-8 -*- import scrapy class AppSpider(scrapy.Spider): name = 'app' allowed_domains = ['google.com'] start_urls = ['http://google.com/'] def parse(self, response): yield scrapy.Request('http://google.com',callback=self.customcallback,meta={ 'data':'test data' }) def customcallback(self,response): received_data = response.meta.get('data') print(received_data) |
