Google Search results JSON missing `related_questions`
complete
Mickael Boudni
Hi,
While search results return "People also ask" questions, the API doesn't return them any more. I confirm it used to work. Maybe a change in SERP html ?
JSON (missing the related_questions): https://serpapi.com/searches/af3be5feaacafb7d/5ff4583276451a9726157a52.json
Thanks for your support
Mickael
Milos Djurdjevic
complete
Milos Djurdjevic
in progress
Elizabeth Oster
planned
Hi Mickael,
When I was reproducing the issue, I was able to get the
related_questions
correctly in the JSON but I can also verify that your instance was wrong.What this can mean is that Google rolled out a change that broke our scraper but has since rolled it back which is why the scraper is correct at the moment.
Regardless, I've documented the issue to our backlog so our engineers will double check on things.
Mickael Boudni
Elizabeth Oster: Hi Elizabeth,
I confirm this is now ok after repeating the same request today. As you said maybe some random styling test from Google.
Thx for your prompt reply anyway.
BR
Mickael
Elizabeth Oster
under review
Mickael Boudni
Elizabeth Oster: Hi Elizabeth,
I came across another example where we don't get the PAA items...
This is pretty key for us as our content suggestion internal tool reports on these.
Could you please have another look?
BR
Mickael
Mickael Boudni
Elizabeth Oster: Here are other examples:
Elizabeth Oster
Mickael Boudni: Thanks for the additional examples!
I've added these examples into our internal documentation as well.
Again, when reproducing your first example, I was able to get the
related_questions
fine but the second example I also had issue with.Our engineers will dig into it further. It's likely that Google has been rolling out some changes that break the scraper and then rolls them back. It's not uncommon for this to happen and it usually is the case when the bug manifests in this way.
Mickael Boudni
Elizabeth Oster: Thx Elizabeth, considering scraping Google search results is the foundation of your business, it'd be great if you could proactively include such "tests" or temporary HTML changes from Google... Especially when multiple examples are shared like this.
Especially when the changes are the same from an example to another.
Thanks for your consideration,
Mickael
Julien Khaleghy
Mickael Boudni: Hi Mickael, we do add HTML and new tests for every situations we encounter. Including the ones you've submitted. Our tests are also run publicly. For example for
related_questions
: https://serpapi.com/related-searches/tests/related-searches/related-searches-spec.rbIt just take a little time to have a fix the issues, add the tests, and deploy a release. Sorry for delays. :/