Skip to main content
Checklist
 

How do I read another website with python? - List

  5.00/5.00 - (1 votes for article, provide comments and help us improve if required.)

To read another website using Python the use of the urllib and beautifulsoup libraries is one option, this way allows you easily to parse the html returned.

The example code below designed to read another website uses a combination of libraries as listed and was deployed in a google container, proven to work.

If trying to achieve the below with wrong versions of code libraries a typical error message lodged would be as follows;
: function object has no attribute build_opener.

How do I read another website with python?

Read another website with Python 3.10.6

## Library versions
## Django==4.1
## beautifulsoup4==4.11.1
## python==3.10.6
from urllib.request import urlopen
from bs4 import BeautifulSoup
# Below code is reading the page title and description if can be found on the web page.               
myTitle=''
myDescription=''
# Prime the page of the website to be read.
myURL='{{insert webpage url here e.g. https://www.example.com}}'
# Setup a request simulating a real user ready to access an external site.
myRequest = urllib.request.Request(url=myURL, headers={'User-Agent':' Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0'})
# Read the page.
external_site_html = urlopen( myRequest ).read()
#
# If the page was found setup a parser to read various elements of interest from the page.
if external_site_html:    
    soup = BeautifulSoup(external_site_html, "html.parser")
    # Read the title and description if present on the webpage
    if soup.title.string:
        myTitle = soup.title.string
    desc = soup.find(attrs={'name':'description'})
    if desc:
            if  desc['content']:
                myDescription=desc['content']
Synonyms/tags: read another website with python, use python to read another external website




Or, per your interest, look at other related links;

  Python
  Beautiful Soup Libraries

Or, why not search for other related lists on our site;

This list was published by Here There, Everywhere and is a printable check-list; To print, hold down the key marked "ctrl", and whilst holding that down, press the "p" key.
This is a downloadable list, select here, if you would like to download the How do I read another website with python? - List as an excel or .csv file. Or find the print function on your browser (top right, three dots, print) and print to a pdf printer if you would like to download as a pdf file, or print to a pdf file.

If you are interested in advertising with us reach out via the contact page below. If you would like to share viewers, and your site is related with great content, then feel free to use the code below on your site then reach out via our contact page and we will look for a place to mention you.


Otherwise, suggest improvements, request additional items, report typos, errors or misspellings, ask us questions or just say hello via the comments here below, we are always pleased to hear from you.

Comments HistoryBe a pioneer and write the first comment.

Note: Current average rating of 5.00 based on 1 reviews and ratings. (1-Low, 5-High)

Note: Comments and ratings help this site get better; if you see something missing, see something wrong, have a question, or want to suggest something to improve then comment below and join the dialogue;

Comments



Looking for something else? Then search the rest of the internet here. If you do like what you see then come back sometime and create your own checklist here. If the item does not have a checkbox next to it, its probably a folder which you can open by clicking. If there are actions listed against an item on the right hand side you can visit websites, download as excel/csv for xls, or create an ical calendar entry.

If you found this page useful, share it, or like us using the buttons on the bottom right, it helps us to improve.

This webpage with information about How do I read another website with python? has the following attributes;
The list was authored by .

« Go look for more free lists & checklists