[DataCap Application] <SXX Future> - <LAMOST DR7 public data>

sxxfuture-official commented 1 year ago

Data Owner Name

LAMOST

What is your role related to the dataset

Data Preparer

Data Owner Country/Region

China

Data Owner Industry

Not-for-Profit

Website

http://www.lamost.org/dr7/v2.0/

Social Media

https://twitter.com/sxxfuture
https://medium.com/@sxxfuture.official
https://discord.gg/GFsJaUFS9K
https://weibo.com/u/7801654981

Total amount of DataCap being requested

15PiB

Expected size of single dataset (one copy)

2PiB

Number of replicas to store

8

Weekly allocation of DataCap requested

600TiB

On-chain address for first allocation

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

[ ] Use Custom Multisig

Identifier

No response

Share a brief history of your project and organization

Focusing on distributed big data storage, disaster data protection, encryption algorithm research, encrypted data product application and Web3.0 new-generation Internet research and development.
SXX Future Data provides government enterprises and individual users with products and service system with data value as the core to meet the ever-expanding demand for mass data storage, management and application.

Is this project associated with other projects/ecosystem stakeholders?

No

If answered yes, what are the other projects/ecosystem stakeholders

No response

Describe the data being stored onto Filecoin

The Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) is a Chinese national scientific research facility operated by the National Astronomical Observatories, Chinese Academy of Sciences. It is a special reflecting Schmidt telescope with 4000 fibers in a field of view of 20 deg2 in the sky. Until July 2019, LAMOST has completed its pilot survey, which was launched in October 2011 and ended in June 2012, and the regular survey of the first seven years, which was initiated on September 2012[1-7]. In this data release, there are totally 10,431,197 low resolution spectra published, which satisfy the selection criteria that the LAMOST LRS General Catalog also used. The data products of this release can be available from the website http://dr7.lamost.org/v2.0/, and they include：

1.    Spectra. - There are 10,431,197 flux (relatively)- and wavelength-calibrated, sky-subtracted spectra in DR7, including 9,846,793 stellar spectra, 198,272 galaxy spectra, 66,612 quasar spectra, and 319,520 unknown object spectra, and these spectra cover the wavelength range of 3690 Å - 9100 Å with a resolution of 1800[2-3] at the 5500 Å.

2.    Spectroscopic Parameter Catalogs. - In this data release, nine spectroscopic parameter catalogs are also published，and they are the LAMOST LRS General Catalog, the LAMOST LRS Stellar Parameter Catalog of A, F, G and K Stars, the LAMOST LRS Line-Index Catalog of A Type Stars, the LAMOST LRS Catalog of gM, dM, and sdM Stars, the LAMOST LRS Multiple Epoch Catalog, the LAMOST LRS Observed Plate Information Catalog, the LAMOST LRS Input Catalog, the LAMOST LRS Catalog of Cataclysmic Variable Stars, and the LAMOST LRS Catalog of  White Dwarf Stars respectively. Tens of parameters are included in these catalogs, such as right ascension, declination, signal to noise ratio (S/N), magnitude, gaia_source_id, gaia_g_mean_mag, atmospheric parameters (effective temperature, surface gravity, and metallicity), radial velocity, element abundance, spectral line indices, line widths, the metallicity sensitive parameter, and the magnetic activity flag.

Where was the data currently stored in this dataset sourced from

Other

If you answered "Other" in the previous question, enter the details here

http://www.lamost.org/dr7/v2.0/tar-split/

If you are a data preparer. What is your location (City and Country)

Hubei Yichang - China

If you are a data preparer, how will the data be prepared? Please include tooling used and technical details?

I will download all the data to the local storage, archive, use the open source and optimized 'singularity' program to process the data, and package the data into an independent car file, which conforms to the sealing format, and will follow up through online and offline sent to SPs distributed in different regions.

If you are not preparing the data, who will prepare the data? (Provide name and business)

No response

Has this dataset been stored on the Filecoin network before? If so, please explain and make the case why you would like to store this dataset again to the network. Provide details on preparation and/or SP distribution.

NO.

Please share a sample of the data

http://www.lamost.org/dr7/v2.0/tar-split/catalog/
http://www.lamost.org/dr7/v2.0/tar-split/lrs-fits/
http://www.lamost.org/dr7/v2.0/tar-split/lrs-png/
http://www.lamost.org/dr7/v2.0/tar-split/mrs-fits/
http://www.lamost.org/dr7/v2.0/tar-split/sky/

Confirm that this is a public dataset that can be retrieved by anyone on the Network

[X] I confirm

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Yearly

For how long do you plan to keep this dataset stored on Filecoin

2 to 3 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, Europe

How will you be distributing your data to storage providers

HTTP or FTP server, Shipping hard drives

How do you plan to choose storage providers

Slack, Partners, Others

If you answered "Others" in the previous question, what is the tool or platform you plan to use

FSPA(FILFI)

If you already have a list of storage providers to work with, fill out their names and provider IDs below

How do you plan to make deals to your storage providers

Boost client, Lotus client

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

large-datacap-requests[bot] commented 1 year ago

Thanks for your request!

Heads up, you’re requesting more than the typical weekly onboarding rate of DataCap!

large-datacap-requests[bot] commented 1 year ago

Thanks for your request! Everything looks good. :ok_hand:

A Governance Team member will review the information provided and contact you back pretty soon.

sxxfuture-official commented 1 year ago

The public time node table of its data. http://www.lamost.org/lmusers/

data-programs commented 1 year ago

This user’s identity has been verified through filplus.storage

Sunnyiscoming commented 1 year ago

Have you prepared enough token for sector pledge?
Best practice for storing large datasets includes ideally, storing it in 3 or more regions, with 4 or more storage provider operators or owners.You should list Miner ID, Business Entity, Location of sps you will cooperate with.
Per the https://github.com/filecoin-project/notary-governance/issues/922 for Open, Public Dataset applicants, please complete the following Fil+ registration form to identify yourself as the applicant and also please add the contact information of the SP entities you are working with to store copies of the data.

This information will be reviewed by Fil+ Governance team to confirm validity and then the application will be triggered for notary review. Let us know if you have any questions.

sxxfuture-official commented 1 year ago

@Sunnyiscoming

Yes, We have.
The relevant information has been submitted through https://submit.jotform.com, please check
Done.

herrehesse commented 1 year ago

@sxxfuture-official I can not clearly see the minerID's in the picture above. Can you post this in text also?

sxxfuture-official commented 1 year ago

Miner IDs	Contact name	SP Organization
f01964215	Xindy	SXX Future
f01986236	Caicai	ChenXing Tec
Uninitial MinerID	Eighteer	KuDian
Uninitial MinerID	Alan	GreaterHeat
Uninitial MinerID	Xindy	SXX Future

ghost commented 1 year ago

@sxxfuture-official let us know when you have 4+ SP entities you are working with. Miner IDs and locations. Thanks

sxxfuture-official commented 1 year ago

@Filplus-govteam

Miner IDs	Contact name	SP Organization	Location
f01964215	Xindy	SXX Future	HongKong
f01986236	Caicai	ChenXing Tec	North America
Uninitial MinerID	Eighteer	KuDian	Greater China
Uninitial MinerID	Alan	GreaterHeat	America
f02195555	Xindy	SXX Future	Greater China

Many minerIDs cannot be determined temporarily, and will not be generated until the project officially starts sealing. More SP entities will join in the future, such as Toploong, NewWebGroup, and ChuangSheng-Cloud.

Sunnyiscoming commented 1 year ago

Datacap Request Trigger

Total DataCap requested

15PiB

Expected weekly DataCap usage rate

600 TiB

Client address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

DataCap allocation requested

300TiB

Id

b3e6ca4f-5c54-4ef7-b215-e0b6d4b8e9a4

1ane-1 commented 1 year ago

Could you send an email to filplus-app-review@fil.org with your official domain in order to confirm your identity?By the way post screenshot here, Thanks a lot.

sxxfuture-official commented 1 year ago

@1ane-1 It`s done.

1ane-1 commented 1 year ago

@1ane-1 It`s done.

Thanks, For the first round,i will support.

1ane-1 commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceatbks3nfrq7mwrcvyam3lhgqg6emxqpzmyyrcwtdq2xu5jagatik

Address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

Datacap Allocated

300.00TiB

Signer Address

f1mdk7s2vntzm6hu35yuo6vjubtrpfnb2awhgvrri

Id

b3e6ca4f-5c54-4ef7-b215-e0b6d4b8e9a4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceatbks3nfrq7mwrcvyam3lhgqg6emxqpzmyyrcwtdq2xu5jagatik

nj-steve commented 1 year ago

As the data is public data, I have some questions. How much data do you have prepared? What's your download speed? Is it enough to seal the data？

sxxfuture-official commented 1 year ago

@nj-steve For this project, the amount of data we have downloaded and formatted as a car file has reached 900T+, and we are still downloading. According to the current data reserve and our download speed, we can start the sealing work.

nj-steve commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceagiitemrb5vecsej4a6onvawb74j46m3crxuimkjhr7vmrdifyvw

Address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

Datacap Allocated

300.00TiB

Signer Address

f1xx6555qijma7igpnjspyvdunc4vfxkawnpqy5ii

Id

b3e6ca4f-5c54-4ef7-b215-e0b6d4b8e9a4

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceagiitemrb5vecsej4a6onvawb74j46m3crxuimkjhr7vmrdifyvw

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

DataCap allocation requested

512TiB

Id

dc154f8e-f843-4845-884d-a4ddba5764de

large-datacap-requests[bot] commented 1 year ago

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

Rule to calculate the allocation request amount

100% weekly > 0.5PiB, requesting 0.5PiB

DataCap allocation requested

512TiB

Total DataCap granted for client so far

300TiB

Datacap to be granted to reach the total amount requested by the client (15PiB)

14.70PiB

Stats

Number of deals	Number of storage providers	Previous DC Allocated	Top provider	Remaining DC
1999	1	300TiB	100	236.90TiB

spaceT9 commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

⚠️ All retrieval success ratios are below 1%.

Overall Graphsync retrieval success rate: 0.00%
Overall HTTP retrieval success rate: 0.00%
Overall Bitswap retrieval success rate: 0.00%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

spaceT9 commented 1 year ago

please increase the retrieval rate before next tranche

sxxfuture-official commented 1 year ago

@spaceT9 Okay, since our project has just started, we will address this aspect in the next two days

sxxfuture-official commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Overall Graphsync retrieval success rate: 22.73%
Overall HTTP retrieval success rate: 0.00%
Overall Bitswap retrieval success rate: 0.00%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

Joss-Hua commented 1 year ago

For the current stage of this LDN, it looks good.

Joss-Hua commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceaotwgpfmcdkvm767es6tc2y3h3db4h3bgqljsbexlbupmq6k5m3m

Address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

Datacap Allocated

512.00TiB

Signer Address

f1tfg54zzscugttejv336vivknmsnzzmyudp3t7wi

Id

dc154f8e-f843-4845-884d-a4ddba5764de

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaotwgpfmcdkvm767es6tc2y3h3db4h3bgqljsbexlbupmq6k5m3m

kernelogic commented 1 year ago

It looks like retrieval ratio has improved. Willing to support.

kernelogic commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacea7gwmayxdjeoarn6nz52usgc3mdpyqyhu27xnmdz6d2rdk2w4sg6

Address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

Datacap Allocated

512.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

dc154f8e-f843-4845-884d-a4ddba5764de

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacea7gwmayxdjeoarn6nz52usgc3mdpyqyhu27xnmdz6d2rdk2w4sg6

sxxfuture-official commented 1 year ago

under sealing

large-datacap-requests[bot] commented 1 year ago

DataCap Allocation requested

Request number 3

Multisig Notary address

f02049625

Client address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

DataCap allocation requested

1PiB

Id

1b4cf216-72dc-4d5e-adb8-1485e54529c5

stcloudlisa commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Overall Graphsync retrieval success rate: 23.42%
Overall HTTP retrieval success rate: 0.00%
Overall Bitswap retrieval success rate: 0.00%

Storage Provider Distribution

⚠️ 1 storage providers have unknown IP location - f02220887

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

stcloudlisa commented 1 year ago

can you tell me why the storage providers have unknown IP location? How will you solve this problem?

sxxfuture-official commented 1 year ago

@stcouldlisa Sorry, this SP has not configure the public IP before, so I have urged the SP to solve it.

lotus net connect f02220887 f02220887 -> {12D3KooWJpUPkQfYK1yyoU1HD5feiYFuzMFnH1kJMchvAJJPsB43: [/ip4/38.98.39.92/tcp/10254]} ERROR: failed to parse multiaddr "f02220887": must begin with /

please check it

ipollo00 commented 1 year ago

LGTM. willing to support.

stcloudlisa commented 1 year ago

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacecgy37godtcwstvgtwwdpt5ekilxcffjgnmwxrzifhhcunepontay

Address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

Datacap Allocated

1.00PiB

Signer Address

f1jvvltduw35u6inn5tr4nfualyd42bh3vjtylgci

Id

1b4cf216-72dc-4d5e-adb8-1485e54529c5

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecgy37godtcwstvgtwwdpt5ekilxcffjgnmwxrzifhhcunepontay

ipollo00 commented 1 year ago

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacecvpkeph5ooprqvwyfvlybpofrqxxhrh4jtzf55de6g6r532l2n4y

Address

f16w2rd5atj3bibqwrjkkdhpskarhcendxonuumcy

Datacap Allocated

1.00PiB

Signer Address

f1n5wlrrhoxpkgwij25xrtt7w7g2k3fhbthmdn6ri

Id

1b4cf216-72dc-4d5e-adb8-1485e54529c5

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacecvpkeph5ooprqvwyfvlybpofrqxxhrh4jtzf55de6g6r532l2n4y

sxxfuture-official commented 1 year ago

checker:manualTrigger

filplus-checker-app[bot] commented 1 year ago

DataCap and CID Checker Report Summary[^1]

Retrieval Statistics

Overall Graphsync retrieval success rate: 26.83%
Overall HTTP retrieval success rate: 0.00%
Overall Bitswap retrieval success rate: 0.00%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard. Click here to view the Retrieval report.

sxxfuture-official commented 1 year ago

under sealing

kevzak commented 1 year ago

SPs taking deals:

f02639702 | Frankfurt am Main, Hesse, DEAmazon.com, Inc. | 62.50 TiB | 6.79% | 62.50 TiB | 0.00% f02230939 | Hong Kong, Central and Western, HKBIH-Global Internet Harbor | 62.50 TiB | 6.79% | 62.50 TiB | 0.00% f02230941 | Hong Kong, Central and Western, HKBIH-Global Internet Harbor | 62.50 TiB | 6.79% | 62.50 TiB | 0.00% f02230375 | Hong Kong, Central and Western, HKBIH-Global Internet Harbor | 62.47 TiB | 6.79% | 62.47 TiB | 0.00% f02220886 | Beijing, Beijing, CNChina Mobile Communications Group Co., Ltd. | 62.16 TiB | 6.75% | 62.16 TiB | 0.00% f02195555 | Beijing, Beijing, CNChina Mobile Communications Group Co., Ltd. | 61.84 TiB | 6.72% | 61.84 TiB | 0.00% f01964215 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 120.81 TiB | 13.12% | 120.81 TiB | 0.00% f02359213 | Shenzhen, Guangdong, CNCHINANET-BACKBONE | 62.34 TiB | 6.77% | 62.34 TiB | 0.00% f02220887new | Los Angeles, California, USCogent Communications | 114.59 TiB | 12.45% | 114.59 TiB | 0.00% f02191111 | Los Angeles, California, USCogent Communications | 62.38 TiB | 6.78% | 62.38 TiB | 0.00% f01986236 | Los Angeles, California, USCogent Communications | 62.16 TiB | 6.75% | 62.16 TiB | 0.00% f090665 | Singapore, Singapore, SGThe Constant Company, LLC | 124.41 TiB | 13.51% | 124.41 TiB | 0.00%

SPs provided upfront:

Seeing two matching SP IDs @sxxfuture-official. Please disclose all SP entities upfront. Last chance

sxxfuture-official commented 1 year ago

herrehesse commented 1 year ago

@sxxfuture-official Who of these is under VPN?

sxxfuture-official commented 12 months ago

checker:manualTrigger

filplus-checker-app[bot] commented 12 months ago

DataCap and CID Checker Report Summary[^1]

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients[^3]

✔️ No CID sharing has been observed.

[^1]: To manually trigger this report, add a comment with text checker:manualTrigger

[^2]: Deals from those addresses are combined into this report as they are specified with checker:manualTrigger

[^3]: To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ...

Full report

Click here to view the CID Checker report. Click here to view the Retrieval Dashboard.

sxxfuture-official commented 11 months ago

checker:manualTrigger

filecoin-project / filecoin-plus-large-datasets