search

Loki-Afro / metalarchives

Unofficial Encyclopedia Metallum API
Other
44 stars 12 forks source link

MetallumException when calling methods (403 Forbidden -- API not working?) #24

Open ari-b-19 opened 5 months ago

ari-b-19 commented 5 months ago

When making method calls to the API, I get an exception thrown: 

Exception in thread "main" com.github.loki.afro.metallum.MetallumException: java.util.concurrent.ExecutionException: java.io.IOException: com.google.api.client.http.HttpResponseException: 403 Forbidden

GET https://www.metal-archives.com/search/ajax-advanced/searching/bands/?bandName=Dreamshade&exactBandMatch=1&genre&themes&location&bandLabelName&indieLabel=0&sEcho=1&iColumns=3&sColumns&iDisplayStart=0&iDisplayLength=200&sNames=,,

    at com.github.loki.afro.metallum.core.util.net.downloader.Downloader.get(Downloader.java:43)
    at com.github.loki.afro.metallum.core.util.net.downloader.Downloader.getHTML(Downloader.java:29)
    at com.github.loki.afro.metallum.search.AbstractSearchService.query(AbstractSearchService.java:86)
    at com.github.loki.afro.metallum.search.AbstractSearchService.access$000(AbstractSearchService.java:18)
    at com.github.loki.afro.metallum.search.AbstractSearchService$1.computeNext(AbstractSearchService.java:65)
    at com.github.loki.afro.metallum.search.AbstractSearchService$1.computeNext(AbstractSearchService.java:54)
    at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
    at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
    at java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1949)
    at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:292)
    at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206)
    at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161)
    at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:298)
    at java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)

It seems like there were some changes to the website based on an update to the unofficial Android app (Metal Empire) but I don't know if that app uses this API or know HTML parsing well enough to understand what may have changed behind the scenes.

ari-b-19 commented 5 months ago

Update: seems to be fine now? maybe just something MA did on the backend

Loki-Afro commented 4 months ago

@ari-b-19 last time i tried to run the tests cloudflare basically was blocking the 4. request or so, might be the same cause if you know any mitigation let me know :/

yes you may get things like:

    at com.github.loki.afro.metallum.search.service.MemberSearchServiceTest.byIdTest(MemberSearchServiceTest.java:36)
Caused by: java.io.IOException: 
com.google.api.client.http.HttpResponseException: 429 Too Many Requests
GET https://www.metal-archives.com/artists/*/1417

:(