Google Scholar Cite search not returning all citation fields
under review
S
Samuel Haysom
I am trying to retrieve citations using the google scholar cite api. In the documentation the returned JSON has fields for MLA, APA, Chicago, Harvard and Vancouver formatted citations. However in my results I am only getting MLA, APA and ISO 690 format citations.
What is causing this discrepancy between the documentation and what I am getting back? I'm also confused about where the ISO 690 citation result comes from as this isn't an option when I use Google Scholar directly
S
Samuel Haysom
Hi Justin O'Hara, I understand that googles results may change, what confuses me is that what I see when I manually access the cite page through google scholar is not always the same as the result from SerpAPI.
As an example, if I manually retrieve results for the search term '"Singer Instruments" ROTOR' (https://scholar.google.com/scholar?q=%22Singer+Instruments%22+ROTOR&hl=en&as_sdt=0%2C5&as_ylo=2000&as_yhi=) and look at result 5 (Development of a new high-throughput method for screening large yeast libraries for use in the beverage industry) I get all citation options (see first screenshot). However if I use the SerpAPI Google Scholar Cite engine I only get MLA, APA and ISO 690 (https://serpapi.com/searches/0780a94de5b191a0/610a67e2f55d77eed03b96cf.json).
It gets odder though because if I click the link returned from "google_scholar_cite_url" I get all the citation options I expect (MLA, APA, Chicago, Harvard, Vancouver, see second screenshot), instead of those in the actual json from SerpAPI.
I don't know if this is because the citation result is location dependent, so that when I access it from my location I get a different set of results to when whatever proxy SerpAPI accesses.
I don't know how this could be solved or if its even solvable. For my use-case I need citation info for each paper returned for a search term. At the moment I am using a citation parser to get that info from the citations on the cite page and to make this reliable as possible I would like to always parse the same citation style. However, this would be unnecessary if SerpAPI could directly return me the bibtex result from the cite page (which it doesn't currently do and Google Scholar is blocking me from doing on my end). I think anyone who is using the SerpAPI Google Scholar Cite API would prefer to be able to retrieve a bibtex file (or one of the other options there) as these are infinitely easier to manipulate programatically and can be used as inputs to other API services such as CrossRef to retrieve more info about papers. Is there any possibility of adding the ability to retrieve bibtex to the SerpAPI Google Scholar Cite API?
Justin O'Hara
Samuel Haysom: I see the discrepancy between the
google_scholar_cite_url
link in the search_metadata
and the scraped results. Thank you for pointing this out.I'm not 100% sure if the ellipsis citation pop-up element is a new javascript rendering or not, so I'm going to add this to the report.
Thanks for reporting this. I hope we can make the changes so you get all the citation fields properly.
Justin O'Hara
Hi Samuel Haysom I apologize for the radio silence. SerpApi is able to scrape exactly what's on the HTML provided by the query. If you are worried that there is going to be different and varying results, unfortunately we deal with Google which is a dynamic engine. Every search may not have all the citation information you need.
If you have an example of a search that is scraping improperly I would love to troubleshoot the issues with you.
Justin O'Hara
under review
S
Samuel Haysom
For my application, I need to consistently be able to access the citation data in one of these formats. My fear is that the fact I am getting different results to the documentation indicates the output from this API won't be consistent over time and I might get a different set of fields on one day versus another. Is this likely to be the case?