harris-chris / Jot.jl

Streamlines the creation and management of AWS Lambda functions written in Julia
MIT License
41 stars 6 forks source link

ArgumentError: Package Jot does not have Pkg in its dependencies #45

Open evanfields opened 1 year ago

evanfields commented 1 year ago

Trying to follow along with the basic example in the readme/docs, an error is thrown. Here's a full terminal session:


(jotting) pkg> st
Status `~/jotting/Project.toml` (empty project)

julia> julia> import Pkg; Pkg.add(url="https://github.com/harris-chris/Jot.jl#main")
ERROR: UndefVarError: julia not defined
Stacktrace:
 [1] top-level scope
   @ REPL[3]:1

(jotting) pkg> st
Status `~/jotting/Project.toml` (empty project)

julia> import Pkg; Pkg.add(url="https://github.com/harris-chris/Jot.jl#main")
    Updating git-repo `https://github.com/harris-chris/Jot.jl#main`
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/jotting/Project.toml`
  [1043c0aa] + Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main#main`
    Updating `~/jotting/Manifest.toml`
  [a8cc5b0e] + Crayons v4.1.1
  [9a962f9c] + DataAPI v1.14.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [59287772] + Formatting v0.4.2
⌃ [cd3eb016] + HTTP v0.9.17
  [83e8ac13] + IniFile v0.5.1
  [ceb4388c] + IsURL v0.2.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [0f8b85d8] + JSON3 v1.12.0
  [1043c0aa] + Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main#main`
  [739be429] + MbedTLS v1.1.7
  [bac558e1] + OrderedCollections v1.6.0
  [9b87118b] + PackageCompiler v2.1.5
  [d96e819e] + Parameters v0.12.3
  [69de0a69] + Parsers v2.5.8
  [21216c6a] + Preferences v1.3.0
⌃ [08abe8d2] + PrettyTables v1.3.1
  [189a3867] + Reexport v1.2.2
  [05181044] + RelocatableFolders v1.0.0
  [6c6a2e73] + Scratch v1.2.0
  [66db9d55] + SnoopPrecompile v1.0.3
  [856f2bd8] + StructTypes v1.10.0
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.10.1
  [5c2747f8] + URIs v1.4.2
  [3a884ed6] + UnPack v1.0.2
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.1
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.0.1+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.0+0
  [14a3606d] + MozillaCACerts_jll v2022.2.1
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [83775a58] + Zlib_jll v1.2.12+3
  [8e850b90] + libblastrampoline_jll v5.1.1+0
  [8e850ede] + nghttp2_jll v1.48.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
        Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> open("increment_vector.jl", "w") do f
         write(f, "increment_vector(v::Vector{Int}) = map(x -> x + 1, v)")
       end
53

julia> using Jot

julia> increment_responder = get_responder("./increment_vector.jl", :increment_vector, Vector{Int})
build_dir = "/tmp/jl_lzzvM8"
  Generating  project increment_vector_package:
    increment_vector_package/Project.toml
    increment_vector_package/src/increment_vector_package.jl
  Activating project at `/tmp/jl_lzzvM8/increment_vector_package`
dependencies = String[]
  Activating project at `~/.julia/environments/v1.8`
LocalPackageResponder{Vector{Int64}}("/home/evan/jotting/increment_vector.jl", :increment_vector, Vector{Int64}, "/tmp/jl_lzzvM8", "increment_vector_package", String[])

julia> local_image = create_local_image(increment_responder; image_suffix="increment-vector")
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 5456k  100 5456k    0     0  6807k      0 --:--:-- --:--:-- --:--:-- 6807k
create_env_script = "using Pkg; cd(\"/tmp/jl_lzzvM8\") do;   Pkg.activate(\".\");   Pkg.add(url=\"https://github.com/harris-chris/Jot.jl\", rev=\"main\");   Pkg.develop(PackageSpec(path=\"./increment_vector_package\"));   Pkg.precompile(); end; "
Status `~/.julia/environments/v1.8/Project.toml` (empty project)
Pkg.status() = nothing
ERROR: ArgumentError: Package Jot does not have Pkg in its dependencies:
- You may have a partially installed environment. Try `Pkg.instantiate()`
  to ensure all packages in the environment are installed.
- Or, if you have Jot checked out for development and have
  added Pkg as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with Jot
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:1167 [inlined]
 [2] macro expansion
   @ ./lock.jl:223 [inlined]
 [3] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [4] eval
   @ ./boot.jl:368 [inlined]
 [5] eval
   @ ~/.julia/packages/Jot/p1f6v/src/Jot.jl:1 [inlined]
 [6] create_environment!(responder::LocalPackageResponder{Vector{Int64}})
   @ Jot ~/.julia/packages/Jot/p1f6v/src/Jot.jl:180
 [7] create_local_image(responder::LocalPackageResponder{Vector{Int64}}; image_suffix::String, aws_config::Nothing, image_tag::String, no_cache::Bool, julia_base_version::String, julia_cpu_target::String, function_test_data::Nothing, package_compile::Bool, user_defined_labels::OrderedCollections.OrderedDict{String, String}, dockerfile_update::Jot.var"#209#213", build_args::OrderedCollections.OrderedDict{String, String})
   @ Jot ~/.julia/packages/Jot/p1f6v/src/Jot.jl:334
 [8] top-level scope
   @ REPL[10]:1

julia> versioninfo()
Julia Version 1.8.5
Commit 17cfb8e65ea (2023-01-08 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 12 × AMD Ryzen 5 1600X Six-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver1)
  Threads: 1 on 12 virtual cores

(@v1.8) pkg> st
Status `~/.julia/environments/v1.8/Project.toml` (empty project)

Note that after the error is thrown, the environment has changed to @v1.8. The same error appears when using Julia 1.9.0-rc1. This is on Windows Subsystem for Linux (I got other errors running from straight Windows, put figure the readme just says tested against Linux so that's where I should run.)

Any suggestions? Am I missing something simple here? Thanks!

harris-chris commented 1 year ago

Hello - thanks for reporting this - I've seen that error before, though I can't quite remember what the cause was. I'm going to be away from my PC all day today but I'll have a look at this tomorrow morning.

On Sat, 1 Apr 2023, 09:25 Evan Fields, @.***> wrote:

Trying to follow along with the basic example in the readme/docs, an error is thrown. Here's a full terminal session:

(jotting) pkg> st Status ~/jotting/Project.toml (empty project)

julia> julia> import Pkg; Pkg.add(url="https://github.com/harris-chris/Jot.jl#main") ERROR: UndefVarError: julia not defined Stacktrace: [1] top-level scope @ REPL[3]:1

(jotting) pkg> st Status ~/jotting/Project.toml (empty project)

julia> import Pkg; Pkg.add(url="https://github.com/harris-chris/Jot.jl#main") Updating git-repo https://github.com/harris-chris/Jot.jl#main https://github.com/harris-chris/Jot.jl#main Updating registry at ~/.julia/registries/General.toml Resolving package versions... Updating ~/jotting/Project.toml [1043c0aa] + Jot v0.1.0 https://github.com/harris-chris/Jot.jl#main#main https://github.com/harris-chris/Jot.jl#main%23main Updating ~/jotting/Manifest.toml [a8cc5b0e] + Crayons v4.1.1 [9a962f9c] + DataAPI v1.14.0 [e2d170a0] + DataValueInterfaces v1.0.0 [59287772] + Formatting v0.4.2 ⌃ [cd3eb016] + HTTP v0.9.17 [83e8ac13] + IniFile v0.5.1 [ceb4388c] + IsURL v0.2.0 [82899510] + IteratorInterfaceExtensions v1.0.0 [0f8b85d8] + JSON3 v1.12.0 [1043c0aa] + Jot v0.1.0 https://github.com/harris-chris/Jot.jl#main#main https://github.com/harris-chris/Jot.jl#main%23main [739be429] + MbedTLS v1.1.7 [bac558e1] + OrderedCollections v1.6.0 [9b87118b] + PackageCompiler v2.1.5 [d96e819e] + Parameters v0.12.3 [69de0a69] + Parsers v2.5.8 [21216c6a] + Preferences v1.3.0 ⌃ [08abe8d2] + PrettyTables v1.3.1 [189a3867] + Reexport v1.2.2 [05181044] + RelocatableFolders v1.0.0 [6c6a2e73] + Scratch v1.2.0 [66db9d55] + SnoopPrecompile v1.0.3 [856f2bd8] + StructTypes v1.10.0 [3783bdb8] + TableTraits v1.0.1 [bd369af6] + Tables v1.10.1 [5c2747f8] + URIs v1.4.2 [3a884ed6] + UnPack v1.0.2 [0dad84c5] + ArgTools v1.1.1 [56f22d72] + Artifacts [2a0f44e3] + Base64 [ade2ca70] + Dates [f43a241f] + Downloads v1.6.0 [7b1f6079] + FileWatching [b77e0a4c] + InteractiveUtils [4af54fe1] + LazyArtifacts [b27032c2] + LibCURL v0.6.3 [76f85450] + LibGit2 [8f399da3] + Libdl [37e2e46d] + LinearAlgebra [56ddb016] + Logging [d6f4376e] + Markdown [a63ad114] + Mmap [ca575930] + NetworkOptions v1.2.0 [44cfe95a] + Pkg v1.8.0 [de0858da] + Printf [3fa0cd96] + REPL [9a3f8284] + Random [ea8e919c] + SHA v0.7.0 [9e88b42a] + Serialization [6462fe0b] + Sockets [fa267f1f] + TOML v1.0.0 [a4e569a6] + Tar v1.10.1 [8dfed614] + Test [cf7118a7] + UUIDs [4ec0a83e] + Unicode [e66e0078] + CompilerSupportLibraries_jll v1.0.1+0 [deac9b47] + LibCURL_jll v7.84.0+0 [29816b5a] + LibSSH2_jll v1.10.2+0 [c8ffd9c3] + MbedTLS_jll v2.28.0+0 [14a3606d] + MozillaCACerts_jll v2022.2.1 [4536629a] + OpenBLAS_jll v0.3.20+0 [83775a58] + Zlib_jll v1.2.12+3 [8e850b90] + libblastrampoline_jll v5.1.1+0 [8e850ede] + nghttp2_jll v1.48.0+0 [3f19e933] + p7zip_jll v17.4.0+0 Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> open("increment_vector.jl", "w") do f write(f, "increment_vector(v::Vector{Int}) = map(x -> x + 1, v)") end 53

julia> using Jot

julia> increment_responder = get_responder("./increment_vector.jl", :increment_vector, Vector{Int}) build_dir = "/tmp/jl_lzzvM8" Generating project increment_vector_package: increment_vector_package/Project.toml increment_vector_package/src/increment_vector_package.jl Activating project at /tmp/jl_lzzvM8/increment_vector_package dependencies = String[] Activating project at ~/.julia/environments/v1.8 LocalPackageResponder{Vector{Int64}}("/home/evan/jotting/increment_vector.jl", :increment_vector, Vector{Int64}, "/tmp/jl_lzzvM8", "increment_vector_package", String[])

julia> local_image = create_local_image(increment_responder; image_suffix="increment-vector") % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 5456k 100 5456k 0 0 6807k 0 --:--:-- --:--:-- --:--:-- 6807k create_env_script = "using Pkg; cd(\"/tmp/jl_lzzvM8\") do; Pkg.activate(\".\"); Pkg.add(url=\"https://github.com/harris-chris/Jot.jl\", rev=\"main\"); Pkg.develop(PackageSpec(path=\"./increment_vector_package\")); Pkg.precompile(); end; " Status ~/.julia/environments/v1.8/Project.toml (empty project) Pkg.status() = nothing ERROR: ArgumentError: Package Jot does not have Pkg in its dependencies:

  • You may have a partially installed environment. Try Pkg.instantiate() to ensure all packages in the environment are installed.
  • Or, if you have Jot checked out for development and have added Pkg as a dependency but haven't updated your primary environment's manifest file, try Pkg.resolve().
  • Otherwise you may need to report an issue with Jot Stacktrace: [1] macro expansion @ ./loading.jl:1167 [inlined] [2] macro expansion @ ./lock.jl:223 [inlined] [3] require(into::Module, mod::Symbol) @ Base ./loading.jl:1144 [4] eval @ ./boot.jl:368 [inlined] [5] eval @ ~/.julia/packages/Jot/p1f6v/src/Jot.jl:1 [inlined] [6] create_environment!(responder::LocalPackageResponder{Vector{Int64}}) @ Jot ~/.julia/packages/Jot/p1f6v/src/Jot.jl:180 [7] create_local_image(responder::LocalPackageResponder{Vector{Int64}}; image_suffix::String, aws_config::Nothing, image_tag::String, no_cache::Bool, julia_base_version::String, julia_cpu_target::String, function_test_data::Nothing, package_compile::Bool, user_defined_labels::OrderedCollections.OrderedDict{String, String}, dockerfile_update::Jot.var"#209#213", build_args::OrderedCollections.OrderedDict{String, String}) @ Jot ~/.julia/packages/Jot/p1f6v/src/Jot.jl:334 [8] top-level scope @ REPL[10]:1

julia> versioninfo() Julia Version 1.8.5 Commit 17cfb8e65ea (2023-01-08 06:45 UTC) Platform Info: OS: Linux (x86_64-linux-gnu) CPU: 12 × AMD Ryzen 5 1600X Six-Core Processor WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-13.0.1 (ORCJIT, znver1) Threads: 1 on 12 virtual cores

@.***) pkg> st Status ~/.julia/environments/v1.8/Project.toml (empty project)

Note that after the error is thrown, the environment has changed to @v1.8. The same error appears when using Julia 1.9.0-rc1. This is on Windows Subsystem for Linux (I got other errors running from straight Windows, put figure the readme just says tested against Linux so that's where I should run.)

Any suggestions? Am I missing something simple here? Thanks!

— Reply to this email directly, view it on GitHub https://github.com/harris-chris/Jot.jl/issues/45, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALDMVSCFFGQGNOAMOJA2RSDW65YYPANCNFSM6AAAAAAWPIH33E . You are receiving this because you are subscribed to this thread.Message ID: @.***>

harris-chris commented 1 year ago

Hello - I'm having trouble re-creating this. Could I check that your jotting project has a source file with a module expression in it? The error you had, Package Jot does not have Pkg in its dependencies, might be the result of having a project that does not have a valid source file.

If that's not the problem, could you try running the example script from within your general julia environment (rather than the environment of the jotting project)?

evanfields commented 1 year ago

Hi, thanks for looking into this.

Could I check that your jotting project has a source file with a module expression in it?

It doesn't -- I'm just following the example from the readme at https://github.com/harris-chris/Jot.jl, so the source file is just increment_vector.jl:

increment_vector(v::Vector{Int}) = map(x -> x + 1, v)

could you try running the example script from within your general julia environment (rather than the environment of the jotting project)?

Sure thing. I followed the readme example exactly, except adding Jot to my @v1.8 environment instead of creating a new environment at ~/jotting. Here's the terminal session:

evan@DESKTOP-12KR94V:~$ mkdir jotting_base_env
evan@DESKTOP-12KR94V:~$ cd jotting_base_env/
evan@DESKTOP-12KR94V:~/jotting_base_env$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.5 (2023-01-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> pwd()
"/home/evan/jotting_base_env"

(@v1.8) pkg> st
Status `~/.julia/environments/v1.8/Project.toml` (empty project)

(@v1.8) pkg> add https://github.com/harris-chris/Jot.jl#main
    Updating git-repo `https://github.com/harris-chris/Jot.jl`
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
    Updating `~/.julia/environments/v1.8/Project.toml`
  [1043c0aa] + Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main`
    Updating `~/.julia/environments/v1.8/Manifest.toml`
  [a8cc5b0e] + Crayons v4.1.1
  [9a962f9c] + DataAPI v1.14.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [59287772] + Formatting v0.4.2
⌃ [cd3eb016] + HTTP v0.9.17
  [83e8ac13] + IniFile v0.5.1
  [ceb4388c] + IsURL v0.2.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [0f8b85d8] + JSON3 v1.12.0
  [1043c0aa] + Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main`
  [739be429] + MbedTLS v1.1.7
  [bac558e1] + OrderedCollections v1.6.0
  [9b87118b] + PackageCompiler v2.1.5
  [d96e819e] + Parameters v0.12.3
  [69de0a69] + Parsers v2.5.8
  [21216c6a] + Preferences v1.3.0
⌃ [08abe8d2] + PrettyTables v1.3.1
  [189a3867] + Reexport v1.2.2
  [05181044] + RelocatableFolders v1.0.0
  [6c6a2e73] + Scratch v1.2.0
  [66db9d55] + SnoopPrecompile v1.0.3
  [856f2bd8] + StructTypes v1.10.0
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.10.1
  [5c2747f8] + URIs v1.4.2
  [3a884ed6] + UnPack v1.0.2
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.1
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.0.1+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.0+0
  [14a3606d] + MozillaCACerts_jll v2022.2.1
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [83775a58] + Zlib_jll v1.2.12+3
  [8e850b90] + libblastrampoline_jll v5.1.1+0
  [8e850ede] + nghttp2_jll v1.48.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
        Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> open("increment_vector.jl", "w") do f
         write(f, "increment_vector(v::Vector{Int}) = map(x -> x + 1, v)")
       end
53

julia> using Jot

julia> increment_responder = get_responder("./increment_vector.jl", :increment_vector, Vector{Int})
build_dir = "/tmp/jl_RTrt0q"
  Generating  project increment_vector_package:
    increment_vector_package/Project.toml
    increment_vector_package/src/increment_vector_package.jl
  Activating project at `/tmp/jl_RTrt0q/increment_vector_package`
dependencies = String[]
  Activating project at `~/.julia/environments/v1.8`
LocalPackageResponder{Vector{Int64}}("/home/evan/jotting_base_env/increment_vector.jl", :increment_vector, Vector{Int64}, "/tmp/jl_RTrt0q", "increment_vector_package", String[])

julia> local_image = create_local_image(increment_responder; image_suffix="increment-vector")
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 5456k  100 5456k    0     0  7563k      0 --:--:-- --:--:-- --:--:-- 7563k
create_env_script = "using Pkg; cd(\"/tmp/jl_RTrt0q\") do;   Pkg.activate(\".\");   Pkg.add(url=\"https://github.com/harris-chris/Jot.jl\", rev=\"main\");   Pkg.develop(PackageSpec(path=\"./increment_vector_package\"));   Pkg.precompile(); end; "
Status `~/.julia/environments/v1.8/Project.toml`
  [1043c0aa] Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main`
Pkg.status() = nothing
  Activating new project at `/tmp/jl_RTrt0q`
    Updating git-repo `https://github.com/harris-chris/Jot.jl`
   Resolving package versions...
    Updating `/tmp/jl_RTrt0q/Project.toml`
  [1043c0aa] + Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main`
    Updating `/tmp/jl_RTrt0q/Manifest.toml`
  [a8cc5b0e] + Crayons v4.1.1
  [9a962f9c] + DataAPI v1.14.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [59287772] + Formatting v0.4.2
⌃ [cd3eb016] + HTTP v0.9.17
  [83e8ac13] + IniFile v0.5.1
  [ceb4388c] + IsURL v0.2.0
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [0f8b85d8] + JSON3 v1.12.0
  [1043c0aa] + Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main`
  [739be429] + MbedTLS v1.1.7
  [bac558e1] + OrderedCollections v1.6.0
  [9b87118b] + PackageCompiler v2.1.5
  [d96e819e] + Parameters v0.12.3
  [69de0a69] + Parsers v2.5.8
  [21216c6a] + Preferences v1.3.0
⌃ [08abe8d2] + PrettyTables v1.3.1
  [189a3867] + Reexport v1.2.2
  [05181044] + RelocatableFolders v1.0.0
  [6c6a2e73] + Scratch v1.2.0
  [66db9d55] + SnoopPrecompile v1.0.3
  [856f2bd8] + StructTypes v1.10.0
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.10.1
  [5c2747f8] + URIs v1.4.2
  [3a884ed6] + UnPack v1.0.2
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [4af54fe1] + LazyArtifacts
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.1
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v1.0.1+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
[+] Building 32.5s (5/14)
 => [internal] load build definition from Dockerfile                                                                                                                   0.1s
 => => transferring dockerfile: 1.08kB                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                      0.1s
 => => transferring context: 2B                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/julia:1.8.4                                                                                                         2.4s
 => [ 1/10] FROM docker.io/library/julia:1.8.4@sha256:127dd5910473df87154481acacf17220fb326f72dbde57cd939b525bf2dbdeda                                                 6.5s
 => => resolve docker.io/library/julia:1.8.4@sha256:127dd5910473df87154481acacf17220fb326f72dbde57cd939b525bf2dbdeda                                                   0.0s
 => => sha256:127dd5910473df87154481acacf17220fb326f72dbde57cd939b525bf2dbdeda 1.25kB / 1.25kB                                                                         0.0s
 => => sha256:38fd7bc589210e13b4ebbd9af4c9dac6ceeb4927f1d9d0eb6226005ef495cf72 1.16kB / 1.16kB                                                                         0.0s
 => => sha256:6848d1eb026fefee534343b75878af849a7d303574b8cf66d94d10f6c8067f9d 5.08kB / 5.08kB                                                                         0.0s
 => => sha256:3f4ca61aafcd4fc07267a105067db35c0f0ac630e1970f3cd0c7bf552780e985 31.40MB / 31.40MB                                                                       0.6s
 => => sha256:1c6a345df56f2d45ba85a9454ae3220bfd6c0fcad5684fc4e21b0702744c651b 2.43MB / 2.43MB                                                                         0.8s
[+] Building 82.3s (15/15) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                   0.1s
 => => transferring dockerfile: 1.08kB                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                      0.1s
 => => transferring context: 2B                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/julia:1.8.4                                                                                                         2.4s
 => [ 1/10] FROM docker.io/library/julia:1.8.4@sha256:127dd5910473df87154481acacf17220fb326f72dbde57cd939b525bf2dbdeda                                                 6.5s
 => => resolve docker.io/library/julia:1.8.4@sha256:127dd5910473df87154481acacf17220fb326f72dbde57cd939b525bf2dbdeda                                                   0.0s
 => => sha256:127dd5910473df87154481acacf17220fb326f72dbde57cd939b525bf2dbdeda 1.25kB / 1.25kB                                                                         0.0s
 => => sha256:38fd7bc589210e13b4ebbd9af4c9dac6ceeb4927f1d9d0eb6226005ef495cf72 1.16kB / 1.16kB                                                                         0.0s
 => => sha256:6848d1eb026fefee534343b75878af849a7d303574b8cf66d94d10f6c8067f9d 5.08kB / 5.08kB                                                                         0.0s
 => => sha256:3f4ca61aafcd4fc07267a105067db35c0f0ac630e1970f3cd0c7bf552780e985 31.40MB / 31.40MB                                                                       0.6s
 => => sha256:1c6a345df56f2d45ba85a9454ae3220bfd6c0fcad5684fc4e21b0702744c651b 2.43MB / 2.43MB                                                                         0.8s
 => => sha256:0c486753e4be770fcbd59861e64ffdde30c61d7422f8d644d3460c3466a5980e 133.10MB / 133.10MB                                                                     2.6s
 => => sha256:c42e9678405e65fe364c52e8f00977ecce31a611af86c85607936a1694e3742b 374B / 374B                                                                             1.1s
 => => extracting sha256:3f4ca61aafcd4fc07267a105067db35c0f0ac630e1970f3cd0c7bf552780e985                                                                              1.3s
 => => extracting sha256:1c6a345df56f2d45ba85a9454ae3220bfd6c0fcad5684fc4e21b0702744c651b                                                                              0.2s
 => => extracting sha256:0c486753e4be770fcbd59861e64ffdde30c61d7422f8d644d3460c3466a5980e                                                                              3.6s
 => => extracting sha256:c42e9678405e65fe364c52e8f00977ecce31a611af86c85607936a1694e3742b                                                                              0.0s
 => [internal] load build context                                                                                                                                      0.1s
 => => transferring context: 5.60MB                                                                                                                                    0.0s
 => [ 2/10] RUN apt-get update && apt-get install -y   build-essential                                                                                                33.0s
 => [ 3/10] RUN mkdir -p /var/runtime/julia_depot                                                                                                                      0.6s
 => [ 4/10] RUN mkdir -p /tmp                                                                                                                                          0.5s
 => [ 5/10] RUN mkdir -p /var/runtime                                                                                                                                  0.5s
 => [ 6/10] WORKDIR /var/runtime                                                                                                                                       0.0s
 => [ 7/10] RUN julia -e "using Pkg; Pkg.Registry.add("General"); "                                                                                                    2.4s
 => [ 8/10] COPY . .                                                                                                                                                   0.1s
 => [ 9/10] RUN julia -e "using Pkg; cd(".") do;   Pkg.activate(".");   Pkg.add(url="https://github.com/harris-chris/Jot.jl", rev="main");   Pkg.develop(PackageSpec  31.0s
 => [10/10] RUN chmod 775 . -R                                                                                                                                         2.9s
 => exporting to image                                                                                                                                                 2.2s
 => => exporting layers                                                                                                                                                2.2s
 => => writing image sha256:e5e7756b7536d147d2eb59aa65f6b66f937c835a2ca41a257cd25772142fffd0                                                                           0.0s
 => => naming to 348967387600.dkr.ecr.us-east-2.amazonaws.com/increment-vector:latest                                                                                  0.0s
LocalImage
  CreatedAt: String "2023-04-02 08:58:32 -0400 EDT"
  Digest: String "<none>"
  ID: String "e5e7756b7536"
  Repository: String "348967387600.dkr.ecr.us-east-2.amazonaws.com/increment-vector"
  Size: String "973MB"
  Tag: String "latest"
  exists: Bool true

The error you had, Package Jot does not have Pkg in its dependencies, might be the result of having a project that does not have a valid source file.

I'm surprised to hear this since the Jot examples/guide show wrapping a simple Julia function in a "naked" script up into a Lambda function, but I will presently test adding all that to a proper package with a UUID generated by ]generate and report back in the following comment in this thread.

evanfields commented 1 year ago

Following up, I can indeed also build the local image for a responder defined within a proper package. Here's the setup, with longer pkg outputs truncated:

evan@DESKTOP-12KR94V:~$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.5 (2023-01-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.8) pkg> generate JottingPackageTest
  Generating  project JottingPackageTest:
    JottingPackageTest/Project.toml
    JottingPackageTest/src/JottingPackageTest.jl

julia> cd("JottingPackageTest/")

(@v1.8) pkg> activate .
  Activating project at `~/JottingPackageTest`

(JottingPackageTest) pkg> add https://github.com/harris-chris/Jot.jl#main
--- pkg outputs, this worked fine ---
  1 dependency successfully precompiled in 1 seconds. 32 already precompiled.

julia> # editing src/JottingPackageTest.jl to include increment_vector

shell> vim src/JottingPackageTest.jl

julia> using Jot

(JottingPackageTest) pkg> precompile
Precompiling project...
  1 dependency successfully precompiled in 1 seconds. 32 already precompiled.

julia> using JottingPackageTest

julia> increment_responder = get_responder(JottingPackageTest, :increment_vector, Vector{Int})
Pinned JottingPackageTest.increment_vector with tree hash 307cf19bfa9b0f23441dfae7ef13c94c29725f03 to /tmp/jl_oogiVo
LocalPackageResponder{Vector{Int64}}("/home/evan/JottingPackageTest", :increment_vector, Vector{Int64}, "/tmp/jl_oogiVo", "JottingPackageTest", String[])

julia> local_image = create_local_image(increment_responder; image_suffix="increment-vector")
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 5456k  100 5456k    0     0  5850k      0 --:--:-- --:--:-- --:--:-- 55.1M
create_env_script = "using Pkg; cd(\"/tmp/jl_oogiVo\") do;   Pkg.activate(\".\");   Pkg.add(url=\"https://github.com/harris-chris/Jot.jl\", rev=\"main\");   Pkg.develop(PackageSpec(path=\"./JottingPackageTest\"));   Pkg.precompile(); end; "
Project JottingPackageTest v0.1.0
Status `~/JottingPackageTest/Project.toml`
  [1043c0aa] Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main`
Pkg.status() = nothing
  Activating new project at `/tmp/jl_oogiVo`
    Updating git-repo `https://github.com/harris-chris/Jot.jl`
   Resolving package versions...
    Updating `/tmp/jl_oogiVo/Project.toml`
  [1043c0aa] + Jot v0.1.0 `https://github.com/harris-chris/Jot.jl#main`
    Updating `/tmp/jl_oogiVo/Manifest.toml`
---more package output truncated---
        Info Packages marked with ⌃ have new versions available and may be upgradable.
   Resolving package versions...
    Updating `/tmp/jl_oogiVo/Project.toml`
  [2c720f12] + JottingPackageTest v0.1.0 `JottingPackageTest`
    Updating `/tmp/jl_oogiVo/Manifest.toml`
  [2c720f12] + JottingPackageTest v0.1.0 `JottingPackageTest`
Precompiling project...
  ✓ JottingPackageTest
  1 dependency successfully precompiled in 1 seconds. 32 already precompiled.
  1 dependency precompiled but a different version is currently loaded. Restart julia to access the new version
[ Info: Temporary environment created
image_name_plus_tag = "348967387600.dkr.ecr.us-east-2.amazonaws.com/increment-vector:latest"
[+] Building 34.1s (15/15) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                   0.0s
 => => transferring dockerfile: 1.04kB                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                        0.0s
 => [internal] load metadata for docker.io/library/julia:1.8.4                                                                                                         0.6s
 => [ 1/10] FROM docker.io/library/julia:1.8.4@sha256:127dd5910473df87154481acacf17220fb326f72dbde57cd939b525bf2dbdeda                                                 0.0s
 => [internal] load build context                                                                                                                                      0.1s
 => => transferring context: 5.61MB                                                                                                                                    0.0s
 => CACHED [ 2/10] RUN apt-get update && apt-get install -y   build-essential                                                                                          0.0s
 => CACHED [ 3/10] RUN mkdir -p /var/runtime/julia_depot                                                                                                               0.0s
 => CACHED [ 4/10] RUN mkdir -p /tmp                                                                                                                                   0.0s
 => CACHED [ 5/10] RUN mkdir -p /var/runtime                                                                                                                           0.0s
 => CACHED [ 6/10] WORKDIR /var/runtime                                                                                                                                0.0s
 => CACHED [ 7/10] RUN julia -e "using Pkg; Pkg.Registry.add("General"); "                                                                                             0.0s
 => [ 8/10] COPY . .                                                                                                                                                   0.0s
 => [ 9/10] RUN julia -e "using Pkg; cd(".") do;   Pkg.activate(".");   Pkg.add(url="https://github.com/harris-chris/Jot.jl", rev="main");   Pkg.develop(PackageSpec  30.4s
 => [10/10] RUN chmod 775 . -R                                                                                                                                         2.6s
 => exporting to image                                                                                                                                                 0.4s
 => => exporting layers                                                                                                                                                0.4s
 => => writing image sha256:9cedac4ea76a66091cd812f9f11d36394f464ba71205107d7c6bdc4d1b361530                                                                           0.0s
 => => naming to 348967387600.dkr.ecr.us-east-2.amazonaws.com/increment-vector:latest                                                                                  0.0s
LocalImage
  CreatedAt: String "2023-04-02 09:27:11 -0400 EDT"
  Digest: String "<none>"
  ID: String "9cedac4ea76a"
  Repository: String "348967387600.dkr.ecr.us-east-2.amazonaws.com/increment-vector"
  Size: String "979MB"
  Tag: String "latest"
  exists: Bool true
harris-chris commented 1 year ago

OK - great - so if I'm understanding this right, it's working OK from within your julia default environment, or if you're using a project you've defined as a responder, but not (as in the original example) if you're trying to use a script as a responder, from within jotter. Is jotter a project (in which case the Project.toml will have a name field near the top), or an environment (with no name field in the Project.toml)? If a project, then julia will be expecting the project to have a src/jotter.jl file, with a module defined in it, I'm wondering if this might be the problem.

evanfields commented 1 year ago

In this case jotting was just an environment, not a project: I did mkdir jotting, cd jotting, then in Julia ]activate ..

OK - great - so if I'm understanding this right, it's working OK from within your julia default environment, or if you're using a project you've defined as a responder, but not (as in the original example) if you're trying to use a script as a responder

Seems right!

harris-chris commented 1 year ago

Yes... when I try it in a fresh environment I get the same result, so it's a legitimate bug, and thanks for reporting this. The bit of code that generates that bug has already been refactored in the branch I'm working on now, so are you OK to use either of the other two (working) methods until that branch is merged in?

evanfields commented 1 year ago

Yep, with two solid work-arounds this isn't at all blocking for me. Thanks for looking into it!