wasmer run doesn't pick the latest package from the registry

yamt commented 10 months ago

Describe the bug

looking at https://wasmer.io/yamt/toywasm as of writing this, the latest version of this package is 34.0.0.

however, for some reasons, wasmer run yamt/toywasm picks and runs 29.0.0.

% wasmer run yamt/toywasm -- --version                           [~]
toywasm v29.0.0
:
:
:

specifying later versions doesn't work either.

wasmer run yamt/toywasm@30 -- --version
error: Unable to find "yamt/toywasm@^30" in the registry
╰─▶ 1: Not found

i tried two versions, one from https://github.com/wasmerio/wasmer/releases/tag/v4.2.3 another from homebrew. the symptom was the same.

wasmer 4.2.3 (7cb550d 2023-10-27)
binary: wasmer-cli
commit-hash: 7cb550db60656352cc527eaf869981d39448e38d
commit-date: 2023-10-27
host: x86_64-apple-darwin
compiler: singlepass,cranelift,llvm

wasmer 4.1.2 (??????? 2023-08-21)
binary: wasmer-cli
commit-hash: 
commit-date: 2023-08-21
host: x86_64-apple-darwin
compiler: cranelift

Steps to reproduce

see above.

Expected behavior

pick the latest version.

Actual behavior

for some reasons it can only see up to 29.0.0.

Additional context

webassembly.sh picks the latest version as expected. https://webassembly.sh/?run-command=toywasm%20--version

Michael-F-Bryan commented 10 months ago

This may be an issue in the resolver code. Possibly the latest() helper or the resolver itself.

maminrayej commented 9 months ago

It seems the issue is due to manifests not being generated for some package versions, causing the runtime to ignore them. The last version with a proper manifest is 29.0.0.

@Michael-F-Bryan I think this issue can be closed here since it's not runtime related.

yamt commented 7 months ago

has this been fixed? i'm still observing the same symptom with wasmer 4.2.5.

theduke commented 7 months ago

@yamt can you provide an example?

You might be seeing a different issue.

yamt commented 7 months ago

@yamt can you provide an example?

You might be seeing a different issue.

spacetanuki% wasmer --version
wasmer 4.2.5
spacetanuki% wasmer run yamt/toywasm@30 -- --version
error: Unable to find "yamt/toywasm@^30" in the registry
╰─▶ 1: Not found
spacetanuki% wasmer run yamt/toywasm@36 -- --version
error: Unable to find "yamt/toywasm@^36" in the registry
╰─▶ 1: Not found
spacetanuki%  wasmer run yamt/toywasm@29 -- --version
toywasm v29.0.0
__clang_version__ = 9.0.0 (https://github.com/llvm/llvm-project 0399d5a9682b3cef71c653373e38890c63c4c365)
__BYTE_ORDER__ is 1234 (__ORDER_LITTLE_ENDIAN__ is 1234)
sizeof(void *) = 4
__wasi__ defined
__wasm__ defined
__wasm32__ defined
Build-time options:
        TOYWASM_USE_SEPARATE_EXECUTE = ON
        TOYWASM_USE_TAILCALL = OFF
        TOYWASM_FORCE_USE_TAILCALL = OFF
        TOYWASM_USE_SIMD = OFF
        TOYWASM_USE_SHORT_ENUMS = ON
        TOYWASM_USE_USER_SCHED = ON
        TOYWASM_ENABLE_TRACING = OFF
        TOYWASM_ENABLE_TRACING_INSN = OFF
        TOYWASM_USE_JUMP_BINARY_SEARCH = ON
        TOYWASM_USE_JUMP_CACHE = OFF
        TOYWASM_JUMP_CACHE2_SIZE = 4
        TOYWASM_USE_LOCALS_CACHE = ON
        TOYWASM_USE_SEPARATE_LOCALS = ON
        TOYWASM_USE_SMALL_CELLS = ON
        TOYWASM_USE_RESULTTYPE_CELLIDX = ON
        TOYWASM_USE_LOCALTYPE_CELLIDX = ON
        TOYWASM_PREALLOC_SHARED_MEMORY = OFF
        TOYWASM_ENABLE_WRITER = ON
        TOYWASM_ENABLE_WASM_SIMD = ON
        TOYWASM_ENABLE_WASM_EXTENDED_CONST = ON
        TOYWASM_ENABLE_WASM_MULTI_MEMORY = ON
        TOYWASM_ENABLE_WASM_TAILCALL = ON
        TOYWASM_ENABLE_WASM_THREADS = ON
        TOYWASM_ENABLE_WASI = ON
        TOYWASM_ENABLE_WASI_THREADS = ON
        TOYWASM_ENABLE_DYLD = ON
spacetanuki%

btw, i published toywasm 36.0.0 yesterday. but for some reasons it isn't shown up on https://wasmer.io/yamt/toywasm.

theduke commented 7 months ago

@ayys please share your findings here.

ayys commented 7 months ago

Thanks for the ping @theduke what I found here is that version 35.0.0 of yamt/toywasm does not compile properly because of the following issue,

Exception: Error while running webc conversion using wapm-to-webc 1.4.3
thread 'main' panicked at 'internal error: entered unreachable code: The path should be fully resolved and relative to the base directory', crates/webc/src/wasmer_package/volume.rs:331:22
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: RuntimeError: unreachable
    at __rust_start_panic (<module>[6197]:0x38fe0b)
    at rust_panic (<module>[6173]:0x38ef43)
    at std::panicking::rust_panic_with_hook::h51603b60b23a0fb8 (<module>[6172]:0x38ef16)
    at std::panicking::begin_panic_handler::{{closure}}::h376f292379b1d694 (<module>[6160]:0x38e035)
    at std::sys_common::backtrace::__rust_end_short_backtrace::h0b174033ce8c7f4e (<module>[6159]:0x38df5f)
    at rust_begin_unwind (<module>[6167]:0x38e5f3)
    at core::panicking::panic_fmt::h0724b0a601bd3679 (<module>[6311]:0x3964ff)
    at webc::wasmer_package::volume::Volume::as_directory_tree::h1dbb74e49d2d352d (<module>[3247]:0x2317da)
    at webc::wasmer_package::package::Package::serialize::he988a23b08eaff5a (<module>[3139]:0x228f71)
    at wapm_to_webc::convert::ConvertOptions::run::ha094318eeeb83961 (<module>[583]:0x5deab)
    at wapm2pirita::main::ha8afd5757497a108 (<module>[213]:0x13c8e)
    at std::sys_common::backtrace::__rust_begin_short_backtrace::hf8a7f79bcbcfb0d2 (<module>[193]:0x12841)
    at std::rt::lang_start::{{closure}}::h938310d90f887e8e (<module>[194]:0x1286a)
    at std::rt::lang_start_internal::hf59d11c495a5d34c (<module>[6008]:0x38393f)
    at __main_void (<module>[215]:0x14c1d)
    at _start (<module>[22]:0x2f78)
╰─▶ 1: RuntimeError: unreachable
           at __rust_start_panic (<module>[6197]:0x38fe0b)
           at rust_panic (<module>[6173]:0x38ef43)
           at std::panicking::rust_panic_with_hook::h51603b60b23a0fb8 (<module>[6172]:0x38ef16)
           at std::panicking::begin_panic_handler::{{closure}}::h376f292379b1d694 (<module>[6160]:0x38e035)
           at std::sys_common::backtrace::__rust_end_short_backtrace::h0b174033ce8c7f4e (<module>[6159]:0x38df5f)
           at rust_begin_unwind (<module>[6167]:0x38e5f3)
           at core::panicking::panic_fmt::h0724b0a601bd3679 (<module>[6311]:0x3964ff)
           at webc::wasmer_package::volume::Volume::as_directory_tree::h1dbb74e49d2d352d (<module>[3247]:0x2317da)
           at webc::wasmer_package::package::Package::serialize::he988a23b08eaff5a (<module>[3139]:0x228f71)
           at wapm_to_webc::convert::ConvertOptions::run::ha094318eeeb83961 (<module>[583]:0x5deab)
           at wapm2pirita::main::ha8afd5757497a108 (<module>[213]:0x13c8e)
           at std::sys_common::backtrace::__rust_begin_short_backtrace::hf8a7f79bcbcfb0d2 (<module>[193]:0x12841)
           at std::rt::lang_start::{{closure}}::h938310d90f887e8e (<module>[194]:0x1286a)
           at std::rt::lang_start_internal::hf59d11c495a5d34c (<module>[6008]:0x38393f)
           at __main_void (<module>[215]:0x14c1d)
           at _start (<module>[22]:0x2f78)

It looks like an issue with LICENSE or README file being improperly set in wasmer.toml.

What can you, the user, do to get this error?

You have two options,

Run wasmer publish --wait. The --wait flag will return an error and not publish the package version if the package cannot be built.
Run wasmer package build before running wasmer publish. This will make sure a package can actually build before we publish it to the registry.

What is `wasmer` doing about this?

We have a PR on the registry that will let users view the errors for corrupted package versions (this will be applied retroactively, so previously published corrupted packages will show up as broken on https://wasmer.io)

We are also discussing how to improve DX on the CLI so users know a package is corrupt before they publish it. The idea is that wasmer publish will try to build the package locally before actually publishing it. This dry-run step will let us know if a package is corrupt or not.

syrusakbary commented 7 months ago

We investigated a bit more the issue:

It seems the issue is that the README and LICENSE are out of the current directory where the manifest is defined:

https://github.com/yamt/toywasm/blob/master/wapm/wapm.toml

The license went from:

license = "BSD-2-Clause"

to:

license-file = "../LICENSE"

We should probably throw a warning if the license file is not found, or readme is not found in the tar.gz

Basically, when using wapm2pirita we should not fail the process of converting from mypackage.tar.gz to mypackage.webc if a file is missing. We should just show a warning, and skip that field on the webc manifest

yamt commented 4 months ago

We investigated a bit more the issue:

It seems the issue is that the README and LICENSE are out of the current directory where the manifest is defined:

https://github.com/yamt/toywasm/blob/master/wapm/wapm.toml

The license went from:
license = "BSD-2-Clause"
to:
license-file = "../LICENSE"
We should probably throw a warning if the license file is not found, or readme is not found in the tar.gz

Basically, when using wapm2pirita we should not fail the process of converting from mypackage.tar.gz to mypackage.webc if a file is missing. We should just show a warning, and skip that field on the webc manifest

readme = "../README.md"
license = "BSD-2-Clause"
license-file = "../LICENSE"

these lines have not been changed since the first version.

wasmerio / wasmer