Closed ZhangChunXian closed 10 months ago
Hi, you can use the data in the file for your research. or you can use the labeled data we released. they both contain the repositories list.
@Zzzzzhuzhiwei I think @ZhangChunXian is requesting the whole repo and user list that OpenDigger export which is not currently in OpenDigger sample data and exported data.
I think we can do this in monthly export task to a csv
or JSONL
files, however the files maybe really large.
I see! If we export this file, it might be too large. Now, there are 328,032,951 different repositories in the clickhouse.
Although there are lots of repos on GitHub but we only export about 500 thousand repos. I try to find out how large the file will be:
If we use csv
file with id
and name
in a line, the file will be about 17MB.
And the user file will be about 5MB, I think this is feasible for monthly export task.
/self-assign
@ZhangChunXian Thanks for the issue, the lists have been exported to repo_list.csv
and user_list.csv
, please feel free to use them in your research. Welcome to any further suggestions and questions.
Usage
For personal research
Extract SQL
I wanna the list of all repositories in the open-digger dataset, in the format "GithubId/repository name, such as "X-lab2017/open-digger". I'd appreciate it if you could provide it. 我想要open-digger数据集中收录的所有仓库的名字, 格式为"githubId/仓库名", 就比如"X-lab2017/open-digger". 如果能提供的话, 万分感激.
Does this dataset need to be updated regularly?
No response