Welcome folks today in this blog post we will be using the selenium library to download the profile picture of the twitter user using the requests module in python. All the full source code of the application will be given below.
Get Started
In order to get started you need to install the below library using the pip command as shown below
pip install selenium
pip install chromedriver_manager
After that you need to make an app.py file and copy paste the following code
app.py
|
1 2 3 4 5 |
from selenium import webdriver import requests from webdriver_manager.chrome import ChromeDriverManager |
As you can see we are importing all the libraries which are needed for this project. Now we need to install the chrome driver for selenium using the webdriver_manager module
|
1 |
driver = webdriver.Chrome(ChromeDriverManager().install()) |
As you can see we are installing the chrome driver automatically by using the install() method. You can even pass a specific version of your chrome browser as the argument.
|
1 2 3 4 5 6 7 |
username = "ronitroy" # navigate to the Twitter profile page driver.get(f'https://twitter.com/{username}/photo') # wait for the page to load driver.implicitly_wait(10) |
As you can see we are providing the twitter username and then we are using the selenium get() method to go to that url and then we are waiting for 10 milliseconds just for the page to be fully loaded.
|
1 2 3 4 5 6 7 8 |
# find the profile image element profile_image = driver.find_element_by_class_name('css-9pa8cd') # get the profile image URL profile_image_url = profile_image.get_attribute('src') # download the profile image response = requests.get(profile_image_url) |
As you can see we are using the selenium library to find the element by their class name and then we are getting the src attribute of that image to the client. And now can use the get() method to go to that url.
|
1 2 3 4 5 6 |
# save the image to a file with open(f'{username}.jpg', 'wb') as f: f.write(response.content) # close the webdriver driver.quit() |
As you can see lastly we are saving the binary data into an actual image jpeg file which will be created in the same directory. And then we are closing the selenium driver.
Full Source Code
Wrapping the blog post this is the full source code of the application here
app.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
from selenium import webdriver import requests from webdriver_manager.chrome import ChromeDriverManager driver = webdriver.Chrome(ChromeDriverManager().install()) username = "ronitroy" # navigate to the Twitter profile page driver.get(f'https://twitter.com/{username}/photo') # wait for the page to load driver.implicitly_wait(10) # find the profile image element profile_image = driver.find_element_by_class_name('css-9pa8cd') # get the profile image URL profile_image_url = profile_image.get_attribute('src') # download the profile image response = requests.get(profile_image_url) # save the image to a file with open(f'{username}.jpg', 'wb') as f: f.write(response.content) # close the webdriver driver.quit() |
