search

Dhaiwat10 / rlp-proxy

The proxy used by @dhaiwat10/react-link-preview to fetch metadata for URLs.
https://www.npmjs.com/package/@dhaiwat10/react-link-preview
34 stars 134 forks source link

Solve the 403 error at a specific URL. #11

Closed JaeseokWoo closed 2 years ago

JaeseokWoo commented 2 years ago

Hello I used "react-link-preview" for my project and confirmed that a 403 error appeared at a specific URL. For example, https://www.coupang.com/. This site seems to be using AkamaiGhost server and returns 403.

In order to solve it, I found that Accept-Encoding should be added to the header. To add Accept-Encodingd, I upgraded the html-metadata-parser version.

I tested it on my local and confirmed that it was solved. But I didn't know why it was solved. I want to help with the "react-link-preview" and want feedback.

I am currently a student, and I am not good at English. That's why the above content is a translator.

Dhaiwat10 commented 2 years ago

Hey, thanks for the pull request. I would accept this instantly but since we are introducing a major / breaking change in the html-metadata-parser dependency, we need to make sure that nothing breaks. Since I have not written any tests for this, I need to test it thoroughly 😅

I will get back to you

JaeseokWoo commented 2 years ago

Thank you for replying. I think it's good to test thoroughly and then apply it. If I have a chance, I want to contribute. Thank you.😄

Dhaiwat10 commented 2 years ago

@JaeseokWoo I tried this out locally and I found a breaking change straight away. If you try to get metadata for https://www.youtube.com/watch?v=5dFa4ijgIPI for example, the old code will give you a proper result but the new one will return undefined and placeholders.

Dhaiwat10 commented 2 years ago

Can you look into why this may be happening? I would love to accept this PR but I also want to make sure none of the existing functionality is broken.

JaeseokWoo commented 2 years ago

Are you talking about undefined/img-placeholder.jpg? I found the cause of this. The cause is url.toLowerCase(). I worked on the no-redis-caching branch, and the v2 logic of the master branch and the no-redis-caching branch is a little different. I removed url.toLowerCase() and tested it, and it seems to be working well. Please check.😄

JaeseokWoo commented 2 years ago

https://github.com/Dhaiwat10/rlp-proxy/pull/8#issue-1003988141

I just checked and the html-metadata-parser version of master branch is 2.0.2. I've been working on the no-redis-caching branch now, but I'd like to apply it to the master branch.

Dhaiwat10 commented 2 years ago

@JaeseokWoo please feel free to create a new branch and PR off of the master branch!

Dhaiwat10 commented 2 years ago

Closing in favor of #12