Inconsistency between Azure Blob Storage and other storage types

[Eitan03]

When storing packages in Azure Blob Storage, the package files are stored under /path/to/module/@v/ but when using other storage types (i.e disk storage), the packages are stored in /path/to/module/$VERSION/ this difference creates problems when trying to change the storage type by uploading an existing folders from the disk to azure

this difference is not documented anywhere (from what Ive seen), when trying to access a package that is stored in blob storage with a disk hierarchy format you recieve 404 without any additional information, and since the logs doesnt describe which path was not found, there isnt any clear way to identify the problem.

To Reproduce Steps to reproduce the behavior:

• Upload a module to blob storage manually (pre filling)
• try to access it via athens

Expected behavior A clear and concise description of what you expected to happen.

Consistent path in all storage platforms, a change is required either to the azure blob storage code to store the packages in /path/to/module/$VERSION/ or to all the other storage types to match azure pathing

Environment (please complete the following information):

• OS: CentOS 7
• Go version : 1.19
• Proxy version : latest ( 0.11.0 )
• Storage (fs/mongodb/s3 etc.) : azure

Additional Information In addition to the difference in the path hierarchy, the file names differ as well, where in disk storage the files are stored as $VERSION.info, source.zip and go.mod, in azure theyre stored as $VERSION.info $VERSION.mod and $VERSION.zip. I assume this difference comes from all of the versions being stored in the same folder in azure and on different folders in other storage types

[DrPsychick]

@Eitan03 one could argue that the internal representation of the storage is not of concern to the user, so I wonder what exactly your use case is. Transferring storage is not in focus of Athens I would say and when it's needed, it is a one-time task that can be easily covered with custom scripting. Or am I wrong?

However, does this issue prevent pre-filling with pacmod? Or how do you "fill" the storage? In other words: what prevents you from "filling" the new storage from scratch?

[DrPsychick]

linking issues #1583

[DrPsychick]

for documentation purposes (when we discuss this), here is a rough overview of code parts:

• Azure: Uses common upload code
• S3: Uses common upload code
• GCP: uses the same as Azure but not the common code
• Mongo: Does it own thing, and thats ok
• Minio: Uses module/version style
• FS: Uses module/version style

so currently, if I'm not mistaken, pacmod works with MinIO and filesystem storage options, not with Cloud provider storage. For an offline mode (some older discussions here: #1538) one should be able to use one of these two.

The argument to leave it as is would be that the internal storage representation of a cloud storage should not be of concern to the user and it could change with future releases. Also, a different storage provider may require a different representation.

[matt0x6F]

We should look at this inconsistency and find a reason for it (and document it) or correct it for version 1.0.0