Quantcast
Channel: Beautiful Soup can't find this html - Stack Overflow
Viewing all articles
Browse latest Browse all 2

Beautiful Soup can't find this html

$
0
0

Python3 - Beautiful Soup 4

I'm trying to parse the weather graph out of the website:https://www.wunderground.com/forecast/us/ny/new-york-city

But when I grab the weather graph html but beautiful soup seems to grab all around it.

I am new to Beautiful Soup. I think it is not able to grab this because either it is not able to parse the tag thing they have going on or because the javascript that populates the graph hasn't loaded or is not parsable by BS (at least the way I'm using it).

As far as my code goes, it's extremely basic

import requests, bs4url = 'https://www.wunderground.com/forecast/us/ny/new-york-city'requrl = requests.get(url, headers={'user-agent': 'Mozilla/5.0'})requrl.raise_for_status()bs = bs4.BeautifulSoup(requrl.text, features="html.parser")a = str(bs)x = 'weather-graph'print(a[a.find('x'):])#Also tried a.find('weather-graph') which returns -1

I have verified that each piece of the code works in other scenarios. The last line should find that string and print out everything after that.

I tried making x many different pieces of the html in and around the graph but got nothing of substance.


Viewing all articles
Browse latest Browse all 2

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>