For most websites, your first requests will always be successful, however, it’s inevitable that some of them will fail. For these failed requests, the API will return a 500 status code and won’t charge you for the request.
In this case, we can make our code retry to make the requests until we reach a maximum number of retries that we set:
from scrapingbee import ScrapingBeeClient # Importing SPB's client
client = ScrapingBeeClient(api_key='YOUR-API-KEY') # Initialize the client with your API Key, and using screenshot_full_page parameter to take a screenshot!
MAX_RETRIES = 5 # Setting the maximum number of retries if we have failed requests to 5.
for _ in range(MAX_RETRIES):
response = client.get("http://scrapingbee.com/blog", params={'screenshot': True}) # Scrape!
if response.ok: # If we get a successful request
with open("./screenshot.png", "wb") as f:
f.write(response.content) # Save the screenshot in the file "screenshot.png"
break # Then get out of the retry loop
else: # If we get a failed request, then we continue the loop
print(response.content)