Not getting data | JSONDecodeError | {"currently_syncing": null}

[kumarvinod002]

Hi I am using tap-jira and target-csv to export jira data hosted locally to csv I have configured config.json for jira and tried both discovery mode and sync mode tap-jira -c config-jira.json -d > catalog.json tap-jira -c config-jira.json -p catalog.json

There are few issues to be noticed here:

1. Even if i run tap-jira in discovery mode leaving all values for each key in config empty { "start_date": "", "username": "", "password": "", "base_url": "", "user_agent": "" } or I run with actual correct values, in both cases I get same catalog.json file and I get no exceptions.
2. when run in sync mode on catalog file (generated with correct values in config) I am getting JSONDecodeError exception:

Traceback (most recent call last): File "C:\Singer\tap-jira\Scripts\tap-jira-script.py", line 33, in sys.exit(load_entry_point('tap-jira==2.0.0', 'console_scripts', 'tap-jira')()) File "c:\singer\tap-jira\lib\site-packages\tap_jira__init.py", line 135, in main raise exc File "c:\singer\tap-jira\lib\site-packages\tap_jira__init.py", line 132, in main main_impl() File "c:\singer\tap-jira\lib\site-packages\tap_jira\init__.py", line 109, in main_impl args = get_args() File "c:\singer\tap-jira\lib\site-packages\tap_jira\init.py", line 28, in get_args unchecked_args = utils.parse_args([]) File "c:\singer\tap-jira\lib\site-packages\singer\utils.py", line 174, in parse_args args.properties = load_json(args.properties) File "c:\singer\tap-jira\lib\site-packages\singer\utils.py", line 109, in load_json return json.load(fil) File "C:\Python39\lib\json\init.py", line 293, in load return loads(fp.read(), File "C:\Python39\lib\json\init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Python39\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Python39\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

3. I can see syncing null {"currently_syncing": null} when trying to run command for sending data to target-csv

   INFO Using Basic Auth API authentication {"currently_syncing": null}

Q1. Is singer compatible with powershell and windows ? Q2. Is there something that I am missing in execution ?

[luandy64]

Even if i run tap-jira in discovery mode leaving all values for each key in config empty { "start_date": "", "username": "", "password": "", "base_url": "", "user_agent": "" } or I run with actual correct values, in both cases I get same catalog.json file and I get no exceptions.

The catalog for jira is built completely from the schema files I believe, so this is not surprising to me

File "c:\singer\tap-jira\lib\site-packages\singer\utils.py", line 174, in parse_args args.properties = load_json(args.properties)

Seems like you have invalid JSON in the catalog made by the tap / in your edits with the metadata

Is singer compatible with powershell and windows ?

No clue. The taps are only written with a *nix environment in mind

[kumarvinod002]

@luandy64 thanks for your response

File "c:\singer\tap-jira\lib\site-packages\singer\utils.py", line 174, in parse_args args.properties = load_json(args.properties)

Seems like you have invalid JSON in the catalog made by the tap / in your edits with the metadata classmethod in catalog.py was

    def load(cls, filename):
        with open(filename) as fp:  # pylint: disable=invalid-name
            return Catalog.from_dict(json.load(fp))

which wasnt working for me in python 3.9 I had to update to

    def load(cls, filename):
        with open(filename,**encoding="utf-16"**) as fp:  # pylint: disable=invalid-name
            return Catalog.from_dict(json.load(fp))

I just edited tap_config.json with my JIRA credentials, urls, start_date and useragent. Then I executed

tap-jira -c config-jira.json -d > catalog.json
tap-jira -c config-jira.json -p catalog.json

noticed in code that -p is depreciated kindly update README.md for same moving ahead I also tried tap-jira -c config-jira.json --catalog catalog.json In that case I am not getting nay RECORD in data.

I didn't edited anything else. Executed with catalog file generated by discovery mode and set

"selected": true

for project type schema in catalog.json

Am I missing something in execution ? Is there something else I need to do ? I am new user It would be kind of you if you can guide me a little here please.

[kumarvinod002]

@luandy64 Context.is_selected(stream.tap_stream_id) is not returning true for any of my streams even if I have "selected": true, added to catalog file

[luandy64]

You should read this guide, I'm not sure you edited the catalog correctly https://github.com/singer-io/getting-started/blob/master/docs/DISCOVERY_MODE.md#example-2

[luandy64]

open(filename,encoding="utf-16")

Seems like this is what is needed to get things working on Windows? I'm not sure what to think here

[kumarvinod002]

@luandy64 thanks for the info on metadata, but in order to get my catalog.json read by tap-jira for sync I had to change tap-jira/init.py from

catalog = Catalog.from_dict(args.properties) \ if args.properties else discover()

to

catalog = args.catalog \ if args.catalog else discover()

Other wise it was expecting -p only and not even accepting -- catalog As per documentation property option is now stale.

also send method in http.py for request needed update for handling failing SSL_VERIFICATION

If its feasible for you could you please check and modify code in your repo also please?

I have got it working with target-csv for stream: projects I am exploring other streams e.g. version, issues etc for which I am seeing some exceptions. I will share the relevant information if required.

Thanks in advance.