search

talsan / ishares

scrape etf holdings histories from iShares.com, with optional aws integration (s3 + athena)
22 stars 9 forks source link

Getting access to AWS history #8

Open MichaelDeBoey opened 5 months ago

MichaelDeBoey commented 5 months ago

Hi @talsan! 👋

I just came across this awesome repo and was wondering if it would be possible to somehow get access to the holdings files on AWS that you already collected?

talsan commented 5 months ago

send me your email address and i can pass over a download file

On Fri, Jun 14, 2024 at 7:55 AM Tal Sansani @.***> wrote:

Hey Michael,

I created this pre-signed url that expires in 12 hrs.

Spans major Us/Intl indices

https://clai-athena-outputs.s3.us-west-2.amazonaws.com/Unsaved/2024/06/14/20c01230-d06e-4105-bd4d-268f35d3b747.csv?response-content-disposition=inline&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEA8aCXVzLXdlc3QtMiJHMEUCIQCSECB837QA9Km8VbwRP46b4SDh9s1eWppcgV4L8lQwnAIgW3aMDDzF%2FaIDATwoZjh%2BIBADqmLqtzqZ3BW7%2BMpoRAMqhAMIqP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FARAAGgwxNDE2MzI3MDMxMjUiDNrpp2exC%2BPUlbk5wyrYArwt6c11Vx%2BlvTvbcdSYX1iVvG6wo2TbKh1xE9DygCGu8Cc1c74sC3Fnhg4QmQNUbVIg1AEul8kZHu5a0BFHpmoUwdqgfxaQdCqC7vGZK6igUe0z8Xi4nHNqx%2Bwqex4XYVJUwc%2BVJNzeS9rVnTr4xzOyxN3Xu%2Fo1zo%2BGllA5a9zczg2ak5Mu10jXrTijjIUQs0OgPhluAxkpXbgLf7uUw783vFuC%2B%2BOt7hIT5Ye4sfR1N3zNowincmGgsC5WikWxhMEOfWlh44Wn6MKYbnEYmZee5EMlHs7EyKzlbYsawV0KUgTD4mK%2FLC7%2BjJ2Av0dZHi17a4rFgi%2BSmwNa6GSxcvARkjh1vVatZkCJNQNyVt22blMpksyL%2F9GWkv3zbYK4C3BeUfFGsbmVaZbziNPztST2lgRoC8wRbi%2BnNhxbqHE7y%2Bjyolrojs%2By4Cw7yWV0mehsA7ex5WmwMK%2BwsbMGOrMCg0Lm2u6WfZ4YHbtUhdMc5EDQhF%2BzodZecGeMnyjDP0IQpdiA%2BZCCDtiaMbyc9sBpBQ3Ub96kcVNrOn1DjoDGni7hssmglO3eQtjenZ7XcrcOLaLoLe4sPp%2BnYvNCMAzh%2FLEPcWUA5O%2BXDIxA5vwhDj5KLNFy3aUYBOf3MUIw4o%2FbzxuHynMxsPbcgFxd3SKk6T5xhZRCAhsbSLUbPXgSaoHP0kyUi4hh0JyAR7q7WwCdqUvbTmbo%2FRLIMrQ6iHt0R%2F1uh5ilKbvYL%2F5KpBe3AaNXWy0FgxFFFuPZ6UApFXlC%2FNBgjIiJh2mWtOED70p8kG2iuzvQ3Er1IUj96nJ4Bug2Nexn3ryKdWp9KZXBRq39qV1UaIczlctxUxv6V7lv4fVQ7JxqDw0lj%2F6WLegsw8Txqw%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20240614T145223Z&X-Amz-SignedHeaders=host&X-Amz-Expires=43199&X-Amz-Credential=ASIASB6PXRKK7JXMNG4C%2F20240614%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=f5ef0913132b129c539a42a6db57c2fd8d56cf218b7d181cf59c5c28055d27db

On Fri, Jun 14, 2024 at 7:05 AM Michaël De Boey @.***> wrote:

Hi @talsan https://github.com/talsan! 👋

I just came across this awesome repo and was wondering if it would be possible to somehow get access to the holdings files on AWS that you already collected?

— Reply to this email directly, view it on GitHub https://github.com/talsan/ishares/issues/8, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNBTUCFUWFDRIXHSNQEKV3ZHL2A5AVCNFSM6AAAAABJKNRYBKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM2TGNBVHE3DMMY . You are receiving this because you were mentioned.Message ID: @.***>

MichaelDeBoey commented 5 months ago

@talsan Thanks! 🙏 You should have received an e-mail

MichaelDeBoey commented 4 months ago

Hey @talsan!

Probably something on my end, but I've send you a couple of e-mails already, but haven't received anything back. Hence I wanted to try it this way to get in touch with you.

If you prefer to talk on Discord and/or Twitter, you can always contact me there as well.

talsan commented 4 months ago

and yeah, industry data providers charge crazy amounts of $...

i have had some success using the ishares data as my core "input data" (it includes very important id info like ticker/exchange/cusip/sedol/etc). and then mixing in something like EOD historical data (around $50-80 a month) or fmp (i think a bit less money and seems solid). iex was a bit more expensive.

this guy had a pretty rich repo plugging into FMP and building an app on top of it: https://github.com/stocknear/backend

On Mon, Jul 8, 2024 at 12:33 PM Michaël De Boey @.***> wrote:

Hey @talsan https://github.com/talsan!

Probably something on my end, but I've send you a couple of e-mails already, but haven't received anything back. Hence I wanted to try it this way to get in touch with you.

If you prefer to talk on Discord and/or Twitter, you can always contact me there as well.

— Reply to this email directly, view it on GitHub https://github.com/talsan/ishares/issues/8#issuecomment-2215016562, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNBTUBG247H2QFFNU7FUQTZLLSSLAVCNFSM6AAAAABJKNRYBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJVGAYTMNJWGI . You are receiving this because you were mentioned.Message ID: @.***>