search

alibaba / clusterdata

cluster data collected from production clusters in Alibaba for cluster management research
1.54k stars 402 forks source link

[cluster-trace-v2018] 404 Errors When Running fetchData.sh Script #216

Closed mpvasilis closed 3 weeks ago

mpvasilis commented 3 weeks ago

I encountered multiple 404 errors while trying to fetch data using the fetchData.sh script on cluster-trace-v2018. The script fails to download several files due to 'Not Found' errors. Below are the details of the errors:

/mnt/d/clusterdata/cluster-trace-v2018$ bash fetchData.sh
fetchData.sh: line 2: $'\r': command not found
fetchData.sh: line 4: $'\r': command not found
mkdir: cannot create directory ‘data\r’: File exists
fetchData.sh: line 7: $'\r': command not found
--2024-06-27 11:21:11--  http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/machine_meta.tar.gz%0D
Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15
Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-06-27 11:21:11 ERROR 404: Not Found.

--2024-06-27 11:21:11--  http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/machine_usage.tar.gz%0D
Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15
Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-06-27 11:21:12 ERROR 404: Not Found.

--2024-06-27 11:21:12--  http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/container_meta.tar.gz%0D
Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15
Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-06-27 11:21:12 ERROR 404: Not Found.

--2024-06-27 11:21:12--  http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/container_usage.tar.gz%0D
Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15
Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-06-27 11:21:13 ERROR 404: Not Found.

--2024-06-27 11:21:13--  http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/batch_task.tar.gz%0D
Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15
Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-06-27 11:21:13 ERROR 404: Not Found.

--2024-06-27 11:21:13--  http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/batch_instance.tar.gz
Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15
Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-06-27 11:21:14 ERROR 404: Not Found.

Here is a draft for opening an issue on the GitHub repository regarding the 404 errors when fetching data using the fetchData.sh script:

Title: 404 Errors When Running fetchData.sh Script

Description: I encountered multiple 404 errors while trying to fetch data using the fetchData.sh script. The script fails to download several files due to 'Not Found' errors. Below are the details of the errors:

mathematica Copy code /mnt/d/clusterdata/cluster-trace-v2018$ bash fetchData.sh fetchData.sh: line 2: $'\r': command not found fetchData.sh: line 4: $'\r': command not found mkdir: cannot create directory ‘data\r’: File exists fetchData.sh: line 7: $'\r': command not found --2024-06-27 11:21:11-- http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/machine_meta.tar.gz%0D Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15 Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2024-06-27 11:21:11 ERROR 404: Not Found.

--2024-06-27 11:21:11-- http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/machine_usage.tar.gz%0D Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15 Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2024-06-27 11:21:12 ERROR 404: Not Found.

--2024-06-27 11:21:12-- http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/container_meta.tar.gz%0D Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15 Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2024-06-27 11:21:12 ERROR 404: Not Found.

--2024-06-27 11:21:12-- http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/container_usage.tar.gz%0D Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15 Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2024-06-27 11:21:13 ERROR 404: Not Found.

--2024-06-27 11:21:13-- http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/batch_task.tar.gz%0D Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15 Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2024-06-27 11:21:13 ERROR 404: Not Found.

--2024-06-27 11:21:13-- http://aliopentrace.oss-cn-beijing.aliyuncs.com/v2018Traces%0D/batch_instance.tar.gz Resolving aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)... 39.97.203.15 Connecting to aliopentrace.oss-cn-beijing.aliyuncs.com (aliopentrace.oss-cn-beijing.aliyuncs.com)|39.97.203.15|:80... connected. HTTP request sent, awaiting response... 404 Not Found 2024-06-27 11:21:14 ERROR 404: Not Found.

Steps to Reproduce:

  1. Clone the repository.
  2. Navigate to the cluster-trace-v2018 directory.
  3. Run the fetchData.sh script using bash fetchData.sh.

Expected Behavior: The script should successfully download all required files without encountering 404 errors.

Actual Behavior: The script fails to download the files and returns 404 errors for each attempted download.

Please investigate this issue and provide a fix or update the URLs if they have changed.

Thank you!