Analysis on file-level Metadata

TSELab / guac-alytics

A series of tools and resources to better understand the risk profile of open source software ecosystems

Apache License 2.0

2 stars 0 forks source link

Analysis on file-level Metadata #50

Open SahithiKasim opened 9 months ago

SahithiKasim commented 9 months ago

Try to match the results with the Debsources paper

[x] Get the matched repositories and versions from the tmp directory(from abhi)
[x] Run the analysis for the command file --mime-type
[x] wc -l

Reference : Debsource paper table 2: Tools used to extract file-level metadata

VinhPham2106 commented 9 months ago

@SahithiKasim we don't have any permissions on most of the files and directories in the /tmp folder. Can you help us out with that?

JorgeH309 commented 9 months ago

@SahithiKasim We ran the cloning script to have a sample of repos that we can work on. Our new question is this: are we running these commands on the repos themselves or on the individual files inside the repos?

SahithiKasim commented 9 months ago

@JorgeH309 Do not clone the repos again. Work on the cloned repos. We need the analysis on repos for specific tags (those are matching with the version from Publish_Packages).

@VinhPham2106 I already gave permissions for everyone to work on clone_repos folder. The path is /data/cyan/guacalytics/raw_data/upstream_clones/clone_repos. Can you tell me what files or directories you need in the /tmp folders and for what so that i can change the permissions accordingly.

VinhPham2106 commented 9 months ago

@SahithiKasim Work done:

Get the script to mostly work (there's the error we're discussing).
json format for the data is set
@JorgeH309 used the generated data to make some visualization
I added scripts to parse the data to gather metrics like mean, median, std_dev, top mime_type, so that ppl with data skills can use them to generate further analytics or visualization.

SahithiKasim commented 8 months ago

@VinhPham2106 and @JorgeH309 are your plots ready from this analysis?