Deprecate partner subdomain

[stupendousC]

• Parent task = Switch scholarphi to new public API endpoint #32158

  • As seen in comment/question, we have 2 goals:
  • GOAL 1 = use the latest graph/v1/ endpoint instead of legacy v1/ endpoint
  • GOAL 2 = use the regular api.semanticscholar.org subdomain, instead of an optional partner.semanticscholar.org subdomain.
    • Question: before, users with S2_PARTNER_API_TOKEN env were calling the partner domain for higher rate limit. Can we use the same token on the regular api.semantic.org subdomain?
    • Rodney says yes.

• THIS PR is for the chi-2021-demo branch.

  • Goal 1 is already done in this PR: #377
  • Goal 2 is the change we're making here.

---

also plz ignore the branch name gh35326_chi2021demo where it referenced the wrong parent ticket lol

TESTING 👍

• Deploy locally, then run pipeline on an example paper.
  • Basically the same steps seen in IDEA 2C in this ticket)
  • Do this WITHOUT a partner API key, logs should show the same.
  • This is prob not necessary but I'm gonna test it out just in case.
  • Yup, I see the same results as the reference ticket above! (The log starting with >>> is temporary and is not in PR)
  • 
  • Do this WITH a partner API key, logs should show that it's being sent to the api.semanticscholar.org subdomain.
  • Lots of existing partner API keys available, I'm just going to borrow [s2-partner-conference-api](https://us-west-2.console.aws.amazon.com/apigateway/home?region=us-west-2#/api-keys/m6l2m75vif) for now.
  • Yup, I see that it's no longer calling the partner subdomain!
  • 
  • Looked at the data-processing/data/citations.json produced and it's legit!

[stupendousC]

Will absorb the totally cool code changes Sam suggested in PR #381 . Will also not leave s2-api.ts alone after all since the main owners are cool with it, it's not like anyone actually uses it anyway, but it's probably best to keep both branches as consistent with each other as possible?

[ca16]

If you're excited to do the api changes too, feel free! Two things about that: 1) I wonder if it would be better to do a separate PR for data-processing code, and a separate PR for api code? What do you think? 2) I don't think I'm equipped to review the api changes. Sounds like maybe it doesn't matter for this case given the review on the PR for main, and the fact that I'm not sure anyone uses the chi-2021-demo branch's api code, but just wanted to make sure that was clear...

[ca16]

Oh also! Still wondering about the testing plan for the data-processing changes...

[stupendousC]

Oh also! Still wondering about the testing plan for the data-processing changes...

Yup, will get on that! stay tuned

[stupendousC]

1. I wonder if it would be better to do a separate PR for data-processing code, and a separate PR for api code? What do you think?
2. I don't think I'm equipped to review the api changes. Sounds like maybe it doesn't matter for this case given the review on the PR for main, and the fact that I'm not sure anyone uses the chi-2021-demo branch's api code, but just wanted to make sure that was clear...

hmmm yeah, I see what you're saying. Ok i'll move the s2-api.ts changes to another PR so it'll be easier to figure out if anything goes wrong on the API side.

[stupendousC]

1. How does the output produced from your runs on this branch compare to output for the same paper produced from a run on the chi-2021-demo branch as it is rn?
2. Did running pytest go alright? (just a reminder there's also that TC job that will run it for you)

	Current chi-2021-demo branch	This branch, WITH API key	This branch, WITHOUT API key
1. compare resulting citations.json	🎯	match 👍	match 👍
2A. pytest	🆗	🆗	🆗
2B. pytest --all locally	fail but 🆗	fail but 🆗	fail but 🆗
2C. pytest --all via TC	🆗	WIP	<--

• Comparing resulting citations.json is a slight headache, bc each obj therein may not be present in the exact same order.
• 2B. pytest --all failed same 7 definition tests on here, but this is a known issue (as previously seen), nothing to do with me.
• 2C. Oops I went with the wrong TC job at first. Thanks @ca16 !!