nats-io / nats-server

High-Performance server for NATS.io, the cloud and edge native messaging system.
https://nats.io
Apache License 2.0
15.93k stars 1.41k forks source link

Unable to subscribe to imported stream in Java #3954

Open randyf2022 opened 1 year ago

randyf2022 commented 1 year ago

I was following the steps in this video (How to set up Decentralized Authentication/Authorization in NATS | Rethink Connectivity Episode 6). I used different account names (SUB/user_sub and PUB/user_pub) as I experimenting with exporting and importing streams. I configured both users for JetStream:

nsc edit account --name SUB --js-mem-storage -1 --js-disk-storage -1 --js-consumer -1 --js-streams -1
nsc edit account --name PUB --js-mem-storage -1 --js-disk-storage -1 --js-streams -1

Using the PUB account I created a stream (FOO with subject BAR) then exported BAR. I ran "nsc describe account" and saw this (sorry for the formatting):

+-----------------------------------------------------------+
|                          Exports                          |
+------+--------+---------+--------+-------------+----------+
| Name | Type   | Subject | Public | Revocations | Tracking |
+------+--------+---------+--------+-------------+----------+
| BAR  | Stream | BAR     | Yes    | 0           | N/A      |
+------+--------+---------+--------+-------------+----------+

Using the SUB account I imported BAR and "aliased" it to BAZ. ran "nsc describe account" and saw this (ditto sorry):

+--------------------------------------------------------------------------+
|                                   Imports                                |
+--------------+--------+--------+-------+---------+--------------+--------+
| Name         | Type   | Remote | Local | Expires | From Account | Public |
+--------------+--------+--------+-------+---------+--------------+--------+
| IMPORTED_BAR | Stream | BAR    | BAZ   |         | PUB          | Yes    |
+--------------+--------+--------+-------+---------+--------------+--------+

I was able to use the nats cli to send a message from PUB to SUB (using the associated credentials). So far so good. Everything is working as expected.

I wrote a little Java application to get a list of streams for PUB and a StreamInfo object was returned for the stream. Again, so far so good.

I then switch to use the credentials for SUB and got back an empty list. Strange that is works for the nats cli but not in Java. Obviously I'm missing something - I just haven't been able to figure it out. Any idea where I ran afoul?

Something to note also - When I run

nats --creds stream ls

I get

No Streams defined.

Which goes along with my Java app to get a list of streams and the count is 0.

I have another Java app where I attempt to subscribe to the imported stream (local is aliases to "BAZ") using the SUB account/user and I get

java.lang.IllegalStateException: [SUB-90007] No matching streams for subject.

Let me know if more specifics are needed.

Thx!

codegangsta commented 1 year ago

Hey @randyf2022, If I'm understanding correctly you are wanting to share a JetStream stream. What about in in particular do you want to share? (there is currently more subjects you need to share for JetStream streams)

randyf2022 commented 1 year ago

@codegangsta, I'm trying to mimic what's going on at my work. My system (SUB above) is wanting to consume data produced by another system (PUB above) that is using several JetStream streams for dissemination of various types of data. The nats cli (using SUB's credentials) shows that SUB has been "imported" the stream (BAR aliases as BAZ) but I'm not able to subscribe to BAZ it via the JetStream client library using SUB's credentials. Here are the cli comments I used:

Export: nsc add export --account PUB --subject BAR

Import: nsc add import -i

? select account SUB ? pick from locally available exports Yes ? select the export -> BAR ? name BAR ? local subject BAZ

After each of the above I did a "nsc push -A".

I executed the import again and got confirmation that the subject was already imported because I got:

Error: account already imports stream "BAR" from

randyf2022 commented 1 year ago

For reference, I'm running Fedora 37 on an HP laptop.

Here are the steps:

export NATS_ENV_NAME=Test nsc add operator --generate-signing-key --sys --name $NATS_ENV_NAME nsc edit operator --require-signing-keys --account-jwt-server-url "nats://0.0.0.0:4222" nsc add account PUB nsc edit account PUB --sk generate nsc add user --account PUB user_pub nsc add account SUB nsc edit account SUB --sk generate nsc add user --account SUB user_sub nsc list keys -A nsc generate config --nats-resolver --sys-account SYS > $NATS_ENV_NAME.conf

In another terminal: ./nats-server --trace --jetstream --config Test.conf

nsc push -A

nsc edit account --name PUB --js-mem-storage -1 --js-disk-storage -1 --js-consumer -1 --js-streams -1 nsc edit account --name SUB --js-mem-storage -1 --js-disk-storage -1 --js-consumer -1 --js-streams -1 nsc push -A

nsc env --account PUB nats --creds $HOME/.local/share/nats/nsc/keys/creds/Test/PUB/user_pub.creds stream add --json --subjects BAR --storage file --replicas 1 --retention limits --discard old --max-msgs=-1 --max-msgs-per-subject=-1 --max-bytes=-1 --max-age 0 --max-msg-size=-1 --dupe-window 2m0s --no-allow-rollup --no-deny-delete --no-deny-purge BAR nats --creds $HOME/.local/share/nats/nsc/keys/creds/Test/PUB/user_pub.creds stream ls

nsc env --account PUB nsc env nsc add export --account PUB --subject BAR nsc push -A

nsc env --account SUB nsc env nsc add import --account SUB --src-account PUB --remote-subject BAR nsc push -A

nats --creds $HOME/.local/share/nats/nsc/keys/creds/Test/PUB/user_pub.creds stream ls

Here's the output:

╭───────────────────────────────────────────────────────────────────────────╮ │ Streams │ ├──────┬─────────────┬─────────────────────┬──────────┬──────┬──────────────┤ │ Name │ Description │ Created │ Messages │ Size │ Last Message │ ├──────┼─────────────┼─────────────────────┼──────────┼──────┼──────────────┤ │ BAR │ │ 2023-03-16 17:56:27 │ 0 │ 0 B │ never │ ╰──────┴─────────────┴─────────────────────┴──────────┴──────┴──────────────╯

nats --creds $HOME/.local/share/nats/nsc/keys/creds/Test/SUB/user_sub.creds stream ls

Output:

No Streams defined

Any help is greatly appreciated.

thangarajks commented 8 months ago

i'm also having the same issue Account A :

Account B :

borjaetxebarria commented 8 months ago

The subjects that an stream captures and the subjects used to interact with the stream are different things - Here you are exporting/importing the subjects captured by the stream (BAR in your case), not the subjects used to interact with the stream. Have a look at this example (it used static authentication, but the idea is the same, you'll see what to export/import): https://github.com/synadia-labs/cross-account-jetstream-sourcing