Closed maxDi84 closed 9 months ago
Thanks for your request! Everything looks good. :ok_hand:
A Governance Team member will review the information provided and contact you back pretty soon.
What is your role at the company that is behind this project? How are you connected to the data set? The website isn't yours. Do you work for the listed organization? How are you finding SPs. List a detailed plan.
@Sunnyiscoming I'm an individual data preparer. Here's no company behind this application. This dataset is public data which is stored in AWS.
There are no restrictions on the use, access, and/or download of data from the End of Term Web Archive Dataset.
These words are from the dataset and the data is public to everyone. The EOT has thus far preserved websites from administration changes. I think it is useful to public. The two SPs we'll work with are recommended by a friend. I'll contact other SPs via slack or other ways.
Total DataCap requested
6PiB
Expected weekly DataCap usage rate
1PiB
Client address
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
f02049625
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
307.19TiB
b5c2bc7e-d1c2-47f4-bfc2-7cf6b0512b39
Can you explain your detailed SP distribution plan?
Hi @maxDi84 For the two sps you mentioned, First one seems cc, second one is a new node. how do you plan to transfer the data to them? Are they able to be reachable? Besides, I remember many sps have sealed "Web Archive Crawl Data", Have you done any research on whether other data preparer has selected this data?
@spaceT9 Our plan about distribution will follow rules 'storage providers should not be storing more than 20% of the duplicated data'. We hope that we can find 4-5 SPs for the first round allocation. Then we will expand the team of cooperation.
@ipollo00 Considering about the distance between us and them, we'll ship hard drives. When we begin the cooperation, they said that they'll ready to be reachable. I don't know who stored this data before.
Can you list the SPs and how about their retrieval success rates?
The initial request for 6 PiB using a newly created account and without establishing trust within the community is concerning. I highly recommend close the application and reconsider by starting with a smaller amount to build credibility and trust before making larger requests.
Advising notaries not to engage.
The initial request for 6 PiB using a newly created account and without establishing trust within the community is concerning. I highly recommend close the application and reconsider by starting with a smaller amount to build credibility and trust before making larger requests.
Advising notaries not to engage.
@herrehesse Hi, I don't know why new accounts are not allowed to apply for Big data sets. I haven't seen this rule in fil+. Maybe you can propose to add it.
@maxDi84 I'm glad you provided such rich data to be included in the filecoin. There are requirements for the number of SP copies, including geographical location. This review has been approved, and if you have any questions during the storage process, please feel free to contact me. Thank you
Your Datacap Allocation Request has been proposed by the Notary
bafy2bzaceajfc3ov4ne472jeqmejc5amaeoa7giusmyjox4xiw5xcc5nevdds
Address
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
Datacap Allocated
307.19TiB
Signer Address
f1cjzbiy5xd4ehera4wmbz63pd5ku4oo7g52cldga
Id
b5c2bc7e-d1c2-47f4-bfc2-7cf6b0512b39
You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceajfc3ov4ne472jeqmejc5amaeoa7giusmyjox4xiw5xcc5nevdds
Welcome more new friends to experience the fil+ program!
Your Datacap Allocation Request has been approved by the Notary
bafy2bzaceavgnatw4hvwafh65pbervmjio522cjetzzguep2frkgj2akp5ine
Address
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
Datacap Allocated
307.19TiB
Signer Address
f174fg3bqbln3zjnkxtyf6s54txqkr7yqkj6cig7y
Id
b5c2bc7e-d1c2-47f4-bfc2-7cf6b0512b39
You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceavgnatw4hvwafh65pbervmjio522cjetzzguep2frkgj2akp5ine
Zero DD by both @Aifabot-Cloud & @zcfil committing to 300T of datacap.
@raghavrmadya @Kevin-FF-USA Please immediately act on this behaviour.
We acknowledge that @maxDi84 did not yet answer a question related to SP entities they are working with above. LINK yet the application was still signed by @Aifabot-Cloud and @zcfil.
Notaries please note that applications can no longer be signed with out clear proof of a distributed storage plan by the client. Max said "they will find 4-5 SPs for the first round". They need to actually find and list the SPs before allocations are made.
Hello @maxDi84 per the new guidelines https://github.com/filecoin-project/notary-governance/issues/922 for Open Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.
This information will be reviewed by Fil+ Governance team to confirm validity toward the Fil+ guideline of a distributed storage plan and then the application will be approved for additional notary review. Let us know if you have any questions.
f02049625
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
512TiB
c2a53ce9-0f48-4c7d-b9f3-9bbe216ba454
f02049625
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
100% weekly > 0.5PiB, requesting 0.5PiB
512TiB
307.18TiB
5.70PiB
Number of deals | Number of storage providers | Previous DC Allocated | Top provider | Remaining DC |
---|---|---|---|---|
3393 | 2 | 307.18TiB | 50.19 | 78.09TiB |
checker:manualTrigger
✔️ Storage provider distribution looks healthy.
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
@Filplus-govteam I have completed the form.
Can you provide distributed storage plan here? There are only 3 sps in the first datacap distribution.
@MRJAVAZHAO Our datacap for the first round is just a little amount. We'll cooperate with more SPs who support retrieval to do much more decentralized distribution. We are communicating with them and it can be 4-6 SPs in next round.
Can you list the name, node, location, contact information?
Yes, here's the SPs we are cooperating with now. f02232088 | Leo | Chengdu, Sichuan f02232007 | Martin | Chengdu, Sichuan f01955030 | Nico | Shanghai
We can confirm this is what was submitted on the registration form as well. No contact information included.
Your Datacap Allocation Request has been proposed by the Notary
bafy2bzacecqtmiynibxilgtnxzz4a53eqavhifzekzvgldjkhi5finpkob5ls
Address
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
Datacap Allocated
512.00TiB
Signer Address
f1sa3dp3a7fwirrsxjdthvzneo7rnjcrrfllsnjpq
Id
c2a53ce9-0f48-4c7d-b9f3-9bbe216ba454
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecqtmiynibxilgtnxzz4a53eqavhifzekzvgldjkhi5finpkob5ls
@maxDi84 would like to see a list of SPs you plan to work with on the next allocation BEFORE it's allocated.
@Filplus-govteam We plan to continue to work with SPs as shown above. Also, we contacted with f02029743 huangjiang, zhejiang; f02233608 Nick ,Hongkong for cooperation on the next allocation. Welcome notaries to help sign for our application!
Dear @Filplus-govteam,
Based on this, @maxDi84 submit the form, and the CID bot and retrieval bot metrics look good. I am willing to sign this.
Besides, pseudonymity is in crypto’s DNA. Have you seen any BTC miner or ETH miner(previously) who was asked for contact information? I am sure there is no contact required when Filecoin starts in 2020,
Thanks
We can confirm this is what was submitted on the registration form as well. No contact information included.
Your Datacap Allocation Request has been approved by the Notary
bafy2bzacedendh63sqmjrsffmnqekyljk6gccghvao2qrjenluzgpoqg426v2
Address
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
Datacap Allocated
512.00TiB
Signer Address
f1yh6q3nmsg7i2sys7f7dexcuajgoweudcqj2chfi
Id
c2a53ce9-0f48-4c7d-b9f3-9bbe216ba454
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedendh63sqmjrsffmnqekyljk6gccghvao2qrjenluzgpoqg426v2
f02049625
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
1PiB
db0415c5-226f-483a-b0fa-6aa6afa95ae7
f02049625
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
200% weekly > 1PiB, requesting 1PiB
1PiB
465661.3YiB
465661.3YiB
Number of deals | Number of storage providers | Previous DC Allocated | Top provider | Remaining DC |
---|---|---|---|---|
15890 | 6 | 512TiB | 25.75 | 130.53TiB |
checker:manualTrigger
✔️ Storage provider distribution looks healthy.
✔️ Data replication looks healthy.
✔️ No CID sharing has been observed.
[^1]: To manually trigger this report, add a comment with text checker:manualTrigger
[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger
[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...
Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.
Good report ,willing to support!
Your Datacap Allocation Request has been proposed by the Notary
bafy2bzacebpi2ccha2tndw6fdquxsjxlebez5manghhbmuo4m5oqczfaxdg3e
Address
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
Datacap Allocated
1.00PiB
Signer Address
f1vxbqrf7rfum3n6m5u6eb4re6xj7amvsaqnzu64y
Id
db0415c5-226f-483a-b0fa-6aa6afa95ae7
You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebpi2ccha2tndw6fdquxsjxlebez5manghhbmuo4m5oqczfaxdg3e
This client is actively stalling http retrievals and blocked http range requests with a reverse proxy to prevent it's data being investigated.
It works as follows:
One set's a bandwidth limit with NGINX on the HTTP retrieval. After a random certain amount the limit is set to zero. This makes the transfer timeout. Because range retrieval is disabled in NGINX one cannot pick up where he left and needs to start all over again.
Log can be found at http://datasetcreators.com/downloadedcarfiles/logs/2078.log
We don't put a bandwidth limit on retrieval. Please don't judge others so easily.
CID Report showing SPs: f02029743 | Hong Kong, Central and Western, HKANLIAN NETWORK TECHNOLOGY CO., LIMITED f01955030 | Shanghai, Shanghai, CNChina Mobile communications corporation f01955034 | Hangzhou, Zhejiang, CNCHINA UNICOM China169 Backbone f02232007 | Chengdu, Sichuan, CNCHINA UNICOM China169 Backbone f02232088 | Chengdu, Sichuan, CNCHINA UNICOM China169 Backbone f02233608 | Chengdu, Sichuan, CNCHINA UNICOM China169 Backbone f02196792 | Hong Kong, Central and Western, HK
What was listed as the SPs: f02232088 | Leo | Chengdu, Sichuan f02232007 | Martin | Chengdu, Sichuan f01955030 | Nico | Shanghai f02029743 huangjiang, zhejiang; f02233608 Nick ,Hongkong
@maxDi84 how is this considered distributed across geographic regions?
@Filplus-govteam Your description is basically accurate; all of these cities are thousands of kilometers away from each other, which fits perfectly with the distribution across geography. Thanks for your comments.
We don't put a bandwidth limit on retrieval. Please don't judge others so easily.
![]()
Ok, let's make this less complicated. It is up to you that your prove that you are storing real data as stated in your LDN. This is done with readily retrieval set the rules and guidelines. I can't retrieve ( See the log ) for whatever reason so we need to fix.
I am trying to receive a sample from you to do due diligence. As can bee seen in the log i am trying to fetch baga6ea4seaqmdrnu2e65kev4jsn3p5f5gjzby2tgsbhykwebwgkpwa5o6ntgafy. This is the sample i want to see.
Please make that piece available for download somewhere so i can retrieve it properly and unpack to check your data.
Let's also ask other notaries to get involved here on retrieval testing. Can you verify cryptowizard's theory.
@cryptowhizzard Ok! You can send a harddisk to me so that I can copy it to you.
We can retrieve it successfully. Please remain and continue to follow the community rules.
@cryptowhizzard Ok! You can send a harddisk to me so that I can copy it to you.
Again, it is your responsibility to have data readily retrievable. It's mine to do due diligence.
Upload the file somewhere so we all can check and download it.
@cryptowhizzard Ok! You can send a harddisk to me so that I can copy it to you.
Again, it is your responsibility to have data readily retrievable. It's mine to do due diligence.
Upload the file somewhere so we all can check and download it.
Once more, we support retrieval and can be downloaded. You do need check for your network and your program. Or you can send a harddisk so that I can copy data to you.
I tried from multiple sources around the globe ( AWS, Google cloud and our own network ) . All same results. I gave you the log as evidence.
Again, you need to make it retrievable without restrictions for everyone on the network, including me.
As alternative i offered you to make the 32 GB available for me on an alternate location but if you refuse your LDN will stay stuck as is.
Data Owner Name
The End of Term Web Archive
What is your role related to the dataset
Data Preparer
Data Owner Country/Region
United States
Data Owner Industry
Government
Website
https://eotarchive.org/
Social Media
Total amount of DataCap being requested
6PiB
Expected size of single dataset (one copy)
630.1TiB
Number of replicas to store
10
Weekly allocation of DataCap requested
1PiB
On-chain address for first allocation
f1aqhgcbhuxhyu2mtiecsrdizz5sx7ptlwa3tgrly
Data Type of Application
Public, Open Dataset (Research/Non-Profit)
Custom multisig
Identifier
No response
Share a brief history of your project and organization
Is this project associated with other projects/ecosystem stakeholders?
No
If answered yes, what are the other projects/ecosystem stakeholders
No response
Describe the data being stored onto Filecoin
Where was the data currently stored in this dataset sourced from
AWS Cloud
If you answered "Other" in the previous question, enter the details here
No response
How do you plan to prepare the dataset
IPFS, lotus, singularity
If you answered "other/custom tool" in the previous question, enter the details here
No response
Please share a sample of the data
Confirm that this is a public dataset that can be retrieved by anyone on the Network
If you chose not to confirm, what was the reason
No response
What is the expected retrieval frequency for this data
Yearly
For how long do you plan to keep this dataset stored on Filecoin
2 to 3 years
In which geographies do you plan on making storage deals
Greater China, Asia other than Greater China, North America, South America, Europe
How will you be distributing your data to storage providers
HTTP or FTP server, IPFS, Shipping hard drives, Lotus built-in data transfer
How do you plan to choose storage providers
Slack, Big Data Exchange, Partners
If you answered "Others" in the previous question, what is the tool or platform you plan to use
No response
If you already have a list of storage providers to work with, fill out their names and provider IDs below
How do you plan to make deals to your storage providers
Lotus client, Singularity
If you answered "Others/custom tool" in the previous question, enter the details here
No response
Can you confirm that you will follow the Fil+ guideline
Yes