search

CAIDA / dzdb-web

DNS Zone DB Frontend Web Application
https://dzdb.caida.org
BSD 2-Clause "Simplified" License
3 stars 1 forks source link

dzdb api test #16

Open bhuffaker opened 3 years ago

bhuffaker commented 3 years ago

@nicole9925 Please create a list of the API end points and a matching call to the web UI.

I suspect most will be a one-to-one mapping.

lanrat commented 3 years ago

Some API endpoints will map to the same web front end.

For example, the info, current, and archive API requests will all map to the same page on the frontend.

nicole9925 commented 3 years ago

Where would be the most convenient place to put this information? Should I put it in the wiki?

bhuffaker commented 3 years ago

Yes. Put it in the wiki.

On Jan 20, 2021, at 3:15 AM, Nicole Lee notifications@github.com wrote:

Where would be the most convenient place to put this information? Should I put it in the wiki?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CAIDA/dzdb-web/issues/16#issuecomment-763532172, or unsubscribe https://github.com/notifications/unsubscribe-auth/AECPT7JNGB4P77ZH5QAHCI3S223LXANCNFSM4WESG3AQ.

nicole9925 commented 3 years ago

I realized that I am not a collaborator on this repository so I cannot make a wiki page. I added the list to my branch in the catalogue data repository. https://github.com/CAIDA/catalog-data/blob/dzdb_v0/sources/recipe/how_to_find_a_domains_ns_and_a_records/dzdb_endpoints.txt

I also tested the end points (using curl)/documentation UI and compared it with the web UI. I wasn't able to test the feed section with the Web UI as I couldn't find the Web UI link, though when I tested curl/documentation different websites showed up for the same calls. Is that normal for it to only display a subset of websites? Otherwise, nothing was out of the ordinary - everything I tested matched.

nicole9925 commented 3 years ago

https://github.com/CAIDA/dzdb-web/wiki/DZDB-API-and-Web-page-Links

I've added it to the wiki.

bhuffaker commented 3 years ago

Please add a link to the wiki to the bottom of the README.

https://github.com/CAIDA/dzdb-web/blob/master/README.md

On Jan 23, 2021, at 3:10 AM, Nicole Lee notifications@github.com wrote:

https://github.com/CAIDA/dzdb-web/wiki/DZDB-API-and-Web-page-Links https://github.com/CAIDA/dzdb-web/wiki/DZDB-API-and-Web-page-Links I've added it to the wiki.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CAIDA/dzdb-web/issues/16#issuecomment-765906701, or unsubscribe https://github.com/notifications/unsubscribe-auth/AECPT7JVYGSYEQPWQMF5JG3S3KVDRANCNFSM4WESG3AQ.

nicole9925 commented 3 years ago

https://github.com/CAIDA/dzdb-web/blob/master/README.md added!

nicole9925 commented 3 years ago

@lanrat Do you need anything else from me?

lanrat commented 3 years ago

@nicole9925 when doing your review of the API, did you find any inconsistencies or bugs that I should be aware of or fix?

nicole9925 commented 3 years ago

@lanrat I didn't find any inconsistencies other than for the feed section, where different domains showed up for the same calls. I wasn't sure if it was because the calls were showing only subsets of a larger list?

lanrat commented 3 years ago

The feeds should not differ. Was is just the order that was different or was the data different as well?

On Mon, Feb 1, 2021 at 3:51 PM Nicole Lee notifications@github.com wrote:

@lanrat https://github.com/lanrat I didn't find any inconsistencies other than for the feed section, where different websites showed up for the same calls. I wasn't sure if it was because the calls were showing only subsets of a larger list?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CAIDA/dzdb-web/issues/16#issuecomment-771242074, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABICYATKQALZVR2HGPH6NLS445APANCNFSM4WESG3AQ .

-- Ian Foster www.vorsk.com

nicole9925 commented 3 years ago

At the time, it was different data. I tried to recreate the problem but it works now. Other than that, I didn't find any inconsistencies.

lanrat commented 3 years ago

That's still odd. I haven't changed anything on my end.

Can you provide the requests you made that differed and the approximate timestamp for when you made them? I'd like to figure out why this happened.

On Wed, Feb 3, 2021, 01:43 Nicole Lee notifications@github.com wrote:

At the time, it was different data. I tried to recreate the problem but it works now. Other than that, I didn't find any inconsistencies.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CAIDA/dzdb-web/issues/16#issuecomment-772374329, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABICYCIQBBSQHAMQITD37DS5ELENANCNFSM4WESG3AQ .

nicole9925 commented 3 years ago

I made them on 12/20. I can't remember the exact calls because I think I tried a few, but I suspect it may have been my error. Sorry for the confusion!

lanrat commented 3 years ago

When is 12/20? I'm not sure how to parse that date.

On Wed, Feb 3, 2021 at 9:30 AM Nicole Lee notifications@github.com wrote:

I made them on 12/20. I think it may have been my error.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CAIDA/dzdb-web/issues/16#issuecomment-772685413, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABICYGAHAVWX7TF23MUED3S5GB4NANCNFSM4WESG3AQ .

-- Ian Foster www.vorsk.com

nicole9925 commented 3 years ago

Sorry I meant I tested it on 1/20/21! 14 days ago (for some reason I thought it was still January).

lanrat commented 3 years ago

Hmm, I don't have anything from then in my logs.

By any change do you know what time of day you observed the discrepancy?

bhuffaker commented 3 years ago

Has asked Nicole to write a script that compares the results of the API call with a canned response. Which should be easier to reproduce the queries.

On Feb 4, 2021, at 9:08 PM, Ian Foster notifications@github.com wrote:

Hmm, I don't have anything from then in my logs.

By any change do you know what time of day you observed the discrepancy?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CAIDA/dzdb-web/issues/16#issuecomment-773791639, or unsubscribe https://github.com/notifications/unsubscribe-auth/AECPT7NEBYJTBRLXPHMVOX3S5N4L5ANCNFSM4WESG3AQ.

nicole9925 commented 3 years ago

I'm not entirely sure about the time, but I made the script! I'll continue testing and see what comes up. Does this look okay?

I read that it would be helpful to use the chai library and mocha framework, so I used that.

https://github.com/CAIDA/catalog-data/blob/dzdb_v0/sources/recipe/how_to_find_a_domains_ns_and_a_records/dzdb_api_test.js

nicole9925 commented 3 years ago

@lanrat Hi Ian! So I updated the test script here as some of the values have changed. Is there anything else you'd like to be added to the script? https://github.com/CAIDA/catalog-data/blob/dzdb_v0/sources/recipe/how_to_find_a_domains_ns_and_a_records/dzdb_api_test.js

I am now getting some discrepancies that I did not get previously. For example, /domains/google.com/nameservers/current should have 4 current nameservers, but returns nothing. The same happens when I input the values on the documentation page.

Here are the values I inputted: domains: google.com ip: 127.0.0.1 nameserver: ns1.google.com zones: net

Screen Shot 2021-02-15 at 3 40 51 PM

lanrat commented 3 years ago

Thanks for pointing this out! Did you also test the .../archive endpoints as well or just .../current?

I added a new filter to the API a few days ago and it must have broken this. This is a perfect example of why I should have more tests...

I'm working on a fix now.

nicole9925 commented 3 years ago

Yes! The archive endpoints passed my tests.

lanrat commented 3 years ago

I just pushed the fix. Let me know how your tests go.

nicole9925 commented 3 years ago

It passes all of the tests! @lanrat

lanrat commented 3 years ago

Thanks for verifying!