rebar3_hex_search:search/3 fails

[AnufrievDE]

Hello,

I've faced an issue when trying to search for any packet with rebar3 hex search. It is fails with functional_clause: [{hex_api_package,search, [{error,no_read_key},<<"recon">>,[]],

It could be related to tricky environment, I am running rebar hex search on Window "Ubuntu" Terminal, https://windowsloop.com/add-ubuntu-to-windows-terminal/

denis_anufriev@HP:/mnt/d/craft_conf_server$ uname -a
Linux HP 4.4.0-18362-Microsoft #1-Microsoft Mon Mar 18 12:02:00 PST 2019 x86_64 x86_64 x86_64 GNU/Linux

Please find the logs below:

denis_anufriev@HP:/mnt/d/craft_conf_server$ DEBUG=1 rebar3 hex search recon
===> 22.0.7 satisfies the requirement for minimum OTP version 18
===> Expanded command sequence to be run: []
===> Provider: {default,do}
===> Expanded command sequence to be run: [{hex,search}]
===> Provider: {hex,search}
xdenanu:#{api_key => undefined,api_organization => undefined,
          api_repository => undefined,api_url => <<"https://hex.pm/api">>,
          http_adapter => r3_hex_http_httpc,
          http_adapter_config => #{profile => rebar},
          http_etag => undefined,
          http_user_agent_fragment =>
              <<"(rebar3/3.12.0+build.4448.refe2ac3217) (httpc)">>,
          name => <<"hexpm">>,repo_key => undefined,repo_name => <<"hexpm">>,
          repo_organization => undefined,
          repo_public_key =>
              <<"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApqREcFDt5vV21JVe2QNB\nEdvzk6w36aNFhVGWN5toNJRjRJ6m4hIuG4KaXtDWVLjnvct6MYMfqhC79HAGwyF+\nIqR6Q6a5bbFSsImgBJwz1oadoVKD6ZNetAuCIK84cjMrEFRkELtEIPNHblCzUkkM\n3rS9+DPlnfG8hBvGi6tvQIuZmXGCxF/73hU0/MyGhbmEjIKRtG6b0sJYKelRLTPW\nXgK7s5pESgiwf2YC/2MGDXjAJfpfCd0RpLdvd4eRiXtVlE9qO9bND94E7PgQ/xqZ\nJ1i2xWFndWa6nfFnRxZmCStCOZWYYPlaxr+FZceFbpMwzTNs4g3d4tLNUcbKAIH4\n0wIDAQAB\n-----END PUBLIC KEY-----">>,
          repo_url => <<"https://repo.hex.pm">>,repo_verify => true,
          repo_verify_origin => true}
xdenanu:{error,no_read_key}
===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace or consult rebar3.crashdump
===> Uncaught error: function_clause
===> Stack trace to the error location:
[{hex_api_package,search,
                  [{error,no_read_key},<<"recon">>,[]],
                  [{file,"/mnt/d/craft_conf_server/_build/default/plugins/hex_core/src/hex_api_package.erl"},
                   {line,43}]},
 {rebar3_hex_search,search,3,
                    [{file,"../src/rebar3_hex_search.erl"},{line,40}]},
 {lists,foreach,2,[{file,"lists.erl"},{line,1338}]},
 {rebar3_hex_search,do,1,[{file,"../src/rebar3_hex_search.erl"},{line,34}]},
 {rebar_core,do,2,
             [{file,"/mnt/d/repos/rebar3/src/rebar_core.erl"},{line,154}]},
 {rebar_prv_do,do_task,5,
               [{file,"/mnt/d/repos/rebar3/src/rebar_prv_do.erl"},{line,87}]},
 {rebar_core,do,2,
             [{file,"/mnt/d/repos/rebar3/src/rebar_core.erl"},{line,154}]},
 {rebar3,run_aux,2,[{file,"/mnt/d/repos/rebar3/src/rebar3.erl"},{line,182}]}]
===> When submitting a bug report, please include the output of `rebar3 report "your command"`
denis_anufriev@HP:/mnt/d/craft_conf_server$ emacs _build/default/plugins/rebar3_hex/src/rebar3_hex_search.erl

Here is a debug changes, if interesting:

-spec do(rebar_state:t()) -> {ok, rebar_state:t()} | {error, string()}.
do(State) ->
    {Args, _} = rebar_state:command_parsed_args(State),
    Term = proplists:get_value(term, Args, ""),
    {ok, Parents} = rebar3_hex_utils:parent_repos(State),
    lists:foreach(fun(Repo) -> **io:format("xdenanu:~p~n",[Repo]),** search(State, Repo, Term) end, Parents),
    {ok, State}.

search(State, Repo, Term) ->
    HexConfig = rebar3_hex_utils:hex_config_read(Repo),
    **io:format("xdenanu:~p~n", [HexConfig]),**
    case hex_api_package:search(HexConfig, rebar_utils:to_binary(Term), []) of
...

[starbelly]

@AnufrievDE hi, I will look into this as soon as possible. Is it only search that fails?

[AnufrievDE]

Hi, I didn't tried anything else except search. Just disabled rebar3_hex pluging in my rebar.config. Would you like me to run anything else?

I've also faced a few problems with rebar3 itself, and after it I decided to reinstall erlang properly(I've used kerl). And hex search has also failed with the same reason after successful re-install, now on erlang 22.1.1.

[starbelly]

@AnufrievDE May be a dumb question... Did you auth using rebar3 hex user auth you can not search hex.pm via the API until you have authorized yourself which is why I ask.

[AnufrievDE]

@starbelly, yes, I didn't. But the issue still applicable for me even after successful authorization.

[starbelly]

@AnufrievDE I see. What versions of erlang, rebar3, and rebar3_hex are you using? I should have asked this to begin with 😁

[AnufrievDE]

erlang/OTP 22.1.1, rebar3 - compiled from sources - e2ac32175806a80a3a92f1e5f554ee7688259350 (Merge pull request #2149 from dmitrivereshchagin/fix-logger-reset)

rebar3_hex - added as plugin to rebar.config, without specifiing a version(probably one of the last).

"rebar3 report hex search pes" output:

`Rebar3 report version 3.12.0+build.4448.refe2ac3217 generated at 2019-10-07T07:01:17+00:00

Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)

Task: hexsearchpes Entered as: hexsearchpes

Operating System: x86_64-unknown-linux-gnu ERTS: Erlang/OTP 22 [erts-10.5.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] Root Directory: /home/denis/kerl/22.1.1 Library directory: /home/denis/kerl/22.1.1/lib

Loaded Applications: bbmustache: 1.6.1 certifi: 2.5.1 cf: 0.2.2 common_test: 1.18 compiler: 7.4.6 crypto: 4.6 cth_readable: 1.4.5 dialyzer: 4.1 edoc: 0.11 erlware_commons: 1.3.1 eunit: 2.3.8 eunit_formatters: 0.5.0 getopt: 1.0.1 hipe: 3.19.1 inets: 7.1 kernel: 6.5 providers: 1.8.1 public_key: 1.7 relx: 3.33.0 sasl: 3.4.1 snmp: 5.4.1 ssl_verify_fun: 1.1.5 stdlib: 3.10 syntax_tools: 2.2.1 tools: 3.2.1

---

Escript path: /mnt/d/repos/rebar3/rebar3 Providers: app_discovery as clean clean compile compile compile cover ct cut deps dialyzer do docs edoc escriptize eunit get-deps help install install_deps key list lock new owner path pkgs publish release relup repo report repos retire revert search shell state tar tree unlock update upgrade upgrade upgrade user version xref `

[geib]

I just started using rebar3_hex and I see the same issue, can't search. I do have hex account, and everything else I try works, which is limited to key list, user stuff, etc. Also reports no "repo" command when I try to list repos.

I have been able to publish and delete to a private hex repo, but search fails against private repo.

I am running rebar 3.12.0 on Erlang/OTP 21 Erts 10.3.5.6, rebar3_hex 6.9.3.

[starbelly]

@AnufrievDE I see. I was able to replicate this for a moment, search consistently failed as did some other ops. I found this was due to bad cache. After I removed _build from a project and re-fetched and recompiled, all works well.

Regardless, I could not replicate searching failing against private repos in general though, here is an example I have:

 $ rebar3 hex search jb_private                                      
Name           Version  Description     URL

jb_private123  0.1.0    An OTP library  https://hex.pm/packages/jb_private123
jb_private123  0.1.0    An OTP library  https://hex.pm/packages/jellybeam/jb_private123

The first entry is public, the second entry is private. If you were to search for this on your end, you would not see the second entry and I can demonstrate that by moving my hex config out of the appropriate location:

 $ mv ~/.config/rebar3/hex.config ~/                                    
 $ rebar3 hex search jb_private                             
Name           Version  Description     URL
jb_private123  0.1.0    An OTP library  https://hex.pm/packages/jb_private123

@geib Could you also provide me with an example search? As for the repo list command, I need to look into this further.

I think a good and recommended solution to at least the cache problems is that we start vendoring in hex_core as suggested by @ericmj . These type of issues happen too frequently it seems and it's always quite confusing. I will see about getting this done in the next few days, until then I would recommend that you make sure you have no

[starbelly]

@geib It looks like the list command for repo was never implemented, we'll get this in as soon as possible.

[geib]

Bryan,

Still no luck.

I removed _build directory in my project, and in ~/.config/rebar3, and also removed ~/.config/rebar3/hex.config.

Then in my project root I did "rebar3 release" to build the erlang release.

Then I did a search like:

rebar3 hex search recon

and I get

===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace or consult rebar3.crashdump ===> When submitting a bug report, please include the output of rebar3 report "your command”

same result if I search for jb_private

The rebar3.crashdump is:

Error: function_clause [{hex_api_package,search, [{error,no_read_key},<<"recon">>,[]], [{file,"/Users/geib/.cache/rebar3/plugins/hex_core/src/hex_api_package.erl"}, {line,45}]}, {rebar3_hex_search,search,3, [{file,"/Users/geib/.cache/rebar3/plugins/rebar3_hex/src/rebar3_hex_search.erl"}, {line,39}]}, {lists,foreach,2,[{file,"lists.erl"},{line,1338}]}, {rebar3_hex_search,do,1, [{file,"/Users/geib/.cache/rebar3/plugins/rebar3_hex/src/rebar3_hex_search.erl"}, {line,34}]}, {rebar_core,do,2, [{file,"/tmp/cirrus-ci-build/src/rebar_core.erl"},{line,154}]}, {rebar_prv_do,do_task,5, [{file,"/tmp/cirrus-ci-build/src/rebar_prv_do.erl"},{line,87}]}, {rebar_core,do,2, [{file,"/tmp/cirrus-ci-build/src/rebar_core.erl"},{line,154}]}, {rebar3,run_aux,2,[{file,"/tmp/cirrus-ci-build/src/rebar3.erl"},{line,182}]}]

Thanks, Mark.

On Feb 12, 2020, at 11:25 PM, Bryan Paxton notifications@github.com wrote:

@AnufrievDE https://github.com/AnufrievDE I see. I was able to replicate this for a moment, search consistently failed as did some other ops. I found this was due to bad cache. After I removed _build from a project and re-fetched and recompiled, all works well.

Regardless, I could not replicate searching failing against private repos in general though, here is an example I have:

$ rebar3 hex search jb_private
Name Version Description URL

jb_private123 0.1.0 An OTP library https://hex.pm/packages/jb_private123 jb_private123 0.1.0 An OTP library https://hex.pm/packages/jellybeam/jb_private123 The first entry is public, the second entry is private. If you were to search for this on your end, you would not see the second entry and I can demonstrate that by moving my hex config out of the appropriate location:

$ mv ~/.config/rebar3/hex.config ~/
$ rebar3 hex search jb_private
Name Version Description URL jb_private123 0.1.0 An OTP library https://hex.pm/packages/jb_private123

@geib Could you also provide me with an example search?

I think a good and recommended solution to at least the cache problems is that we start vendoring in hex_core as suggested by @ericmj . These type of issues happen too frequently it seems and it's always quite confusing. — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tsloughter/rebar3_hex/issues/136?email_source=notifications&email_token=AAAKT6SG4GRZ3F6D7UBWVVLRCTRVXA5CNFSM4I3CWF62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELTR6FQ#issuecomment-585572118, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAKT6UND6KDPRQAXXRNSXLRCTRVXANCNFSM4I3CWF6Q.

[starbelly]

@geib Is this still an issue for you?

[geib]

Yes, I believe it is.

I have an instance of the hexpm mini-repo running and when using elixir hex commands everthing seems to work as advertised including search. However I am still unable to use search from rebar3 with the rebar3_hex plugin. I am in the middle of testing at the moment so I may be able to give more information later this week.

Thanks,

On Aug 16, 2020, at 2:35 PM, Bryan Paxton notifications@github.com wrote:

@geib https://github.com/geib Is this still an issue for you?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tsloughter/rebar3_hex/issues/136#issuecomment-674574637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAKT6U7NRSPWEIEZCMGPRDSBA7INANCNFSM4I3CWF6Q.

[geib]

Update:

I have been able to get “rebar3 hex search ” to work by first executing "rebar3 hex user auth”. I had two repos defined in global rebar.config, after removing the local repo auth took effect. With two repos I don’t think the auth against hex.pm was working.

Once I was able to authenticate search works as expected.

Thanks, Mark.

On Aug 16, 2020, at 2:35 PM, Bryan Paxton notifications@github.com wrote:

@geib https://github.com/geib Is this still an issue for you?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tsloughter/rebar3_hex/issues/136#issuecomment-674574637, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAKT6U7NRSPWEIEZCMGPRDSBA7INANCNFSM4I3CWF6Q.

[starbelly]

@geib Great! Closing this as resolved ❤️

[jechol]

@AnufrievDE May be a dumb question... Did you auth using rebar3 hex user auth you can not search hex.pm via the API until you have authorized yourself which is why I ask.

Thanks! This helps my issues.