jetperch / pyjoulescope_ui

Joulescope graphical user interface
https://www.joulescope.com
Apache License 2.0
85 stars 26 forks source link

[Feature Request] - Logic Saleae data import #98

Open hh-decr opened 4 years ago

hh-decr commented 4 years ago

My main use case with this device is to have two devices always connected.

I would like to be able to import some digital (and analog?) waveforms. For me, supporting just the Saleae Logic is fine. Is there any expected support path for this?

(Love the Joulescope so far! I have wanted this exact thing for so long!)

mliberty1 commented 4 years ago

Hi @gideon-decr

Great to hear that you are loving your Joulescope so far!

We do not currently have plans to integrate with Saleae Logic directly. The original ".logicdata" sounds messy: https://support.saleae.com/faq/technical-faq/can-i-create-or-edit-logicdata-files

They do have a feature request to document the new ".sal" Logic 2 format: https://saleae.upvoty.com/b/feature-requests/provide-info-on-the-sal-file-format/

We have thought about integrating with sigrok, which is open-source. What do you think?

Which is more important to you? Exporting Joulescope UI data to another tool or importing the other tool's data into the Joulescope UI? If you don't care, that works, too ;)

hh-decr commented 3 years ago

It is your program! Support whatever you want. Just wanted to let you know what I would use most.

What I really want is a workflow to go from two signals to a synced set of signals. I don't care if I have to export it and then run it through a script and then import it.

Ideally, you would define a synchronization scheme that would simultaneously sync the two signals and sync the relative timing.

Example: embedded code: static pseudo random sequence of us delays and toggling the bit each time (this lets you use a match filter to find the location of these easily and uniquely ) A series of toggles with 1us between each

The option to have another of that toggle sequence at the "end" of the interesting area of the signal. (Or just every X period of time)

You can use the location of these sequences in the digital signal from the Joulescope and the same signal from the other device (ex. Saleae) to then resample the other devices signals to make them all line up. (This matters more and more the longer the sampling period.

Another request you mentioned recently talked about multi-day recordings. That kind of recording would be the type of thing that would really shine. The waveform from the Joulescope and the other device might be wildly off by the end. (Perhaps by minutes!)

I might have some code that solves this problem. Contact me if you decide you want to tackle this problem and I can share it with you privately.

mliberty1 commented 3 years ago

At one point in a past life, I used a 50% duty cycle pulse-per-second signal. The rising edge of the pulse was accurately triggered, such as by a GPS receiver. The falling edge was then followed by the UART-encoded timestamp of the rising edge of the pulse. This presented a unique signal for doing just this type of resynchronization.

Your Joulescope has ±20 ppm accuracy. Another option is to correct the sampling rate to your PC's clock, which is usually synchronized through NTP, which is based upon very accurate Stratum 1 servers. This would significantly reduce the amount of clock scaling issues. It would also help with #55.

hh-decr commented 3 years ago

Oh, sorry if it seemed if I was attacking the Joulescope. I am not. It doesn't matter if the Joulescope is perfectly accurate. The device connected to the GPIO pin likely has a clock that will have an offset and drift over time. (both faster AND slower) Pushing synchronization to the embedded device isn't really a good idea. Introducing a third party into the mix won't help a lot in this case. (because then you have to prove how often your Joulescope is being synced to that same party's clock) You want your two signal streams to be aligned and resampled to be on the same time basis relative to one another. And, ideally, you probably don't want either of them synched during the process to cause a likely time-basis jump in one system.

The non-constant-width-square-wave-ness of the signal I was talking about allows you to use that channel for lots of other signals without concern of confusion. The fact that you only have two GPIO inputs (and the embedded system might only have one GPIO output!) means that it is important to consider unambiguous in-band synchronization techniques.

I know this is an advanced feature and perhaps something to add to an issue to get back to ... never. But it might at least be worth thinking about how to expose the needed tools to others to make it easy for them to put together a system like this. (Perhaps they already exist and I haven't thought hard enough about it yet. Or it simply requires exporting and not using your software for visualization.)

hh-decr commented 3 years ago

I still think it would be a game changer for you to define the scheme and have your program support it but perhaps this is possible via this process: Ability to export just the GPIO waveforms from the jls file with time stamp information. (Perhaps this already exists. I haven't looked yet.) A script is run separately that takes the GPIO waveform and the external signals and resamples them to be on the time basis of the jls file. You support importing a logic-analyzer type file format for bringing those back into the Joulescope program. (Ideally this import would allow text overlay of some sort to allow import of protocol decoder's information on logic streams.) All signals are now displayed in the Joulescope program on the same time-basis with all of the interesting current and logic-decoded information in one place.

That makes it such that the supported features for you are minimal and probably useful in 1000s of other contexts. And you just work with someone (possibly myself) to create a tutorial on this particular workflow along with the scripts that solve the hard synching problems.

mliberty1 commented 3 years ago

@gideon-decr Oh, sorry if it seemed if I was attacking the Joulescope. I am not.

No worries! I didn't take it that way. Unless you use a precision lab clock for everything, devices will drift. I do need to figure out how to match accurate UTC time onto Joulescope samples, so that's likely the first approach.

@gideon-decr Ability to export just the GPIO waveforms from the jls file with time stamp information. (Perhaps this already exists. I haven't looked yet.)

The existing 0.9.2 Joulescope UI does not have this feature, but it would be a reasonably easy feature, once we have UTC time.

@gideon-decr You support importing a logic-analyzer type file format for bringing those back into the Joulescope program.

I really like the Joulescope UI for current, charge, voltage, power, and energy analysis, but am a little concerned about the amount of software work here as the Joulescope UI is not even close to being a full-fledged logic analyzer. It's a long way from displaying any protocol decoder info. We may be able to add sigrok integration or exporting so that we don't have to recreate everything from scratch.

hh-decr commented 3 years ago

Last thing I will say about this: If you don't want to support the data streams in your app then you should create the ability to import a time synchronization vector. (This would be conserved along with the UTC sync approach.)

Just allow the import of 2 1d vectors that create a mapping from sample number to time stamp (or any arbitrary integer). Then all you have to do to support this is create the function for mapping from your integers to the other integer. (That is just a curve fitting operation to obtain a simple mathematical function to go from counts to time.) You would actually only have to run the integers that you display on your UI or transform the integers on export using this mapping function. You technically wouldn't have to change anything else in your UI. (I am sure there are 1-2 other places you would have to map back and fourth.) You then can conditionally apply a int->int -> timestamp string transformation for the case of going to a easy to read human string in place of an integer.

Then you just need to store in your JLS file an occasional set of times (perhaps once every 10 minutes and when it is started and stopped) when you sent an integer from the Joulescope and had a high degree of reliability on time received on the host. (This could be via an explicit sync operation or just when you happen to know the last sample was in a stream operation.)

Then you can use the same int->int approach for timestamps for a while and then add the int->int->timestamp when you have time to tune the UI to handle all the strange text formatting corner cases. You could even just do an counts_int->fxp_utc_int->utc_float for a while so you can go from counts to utc in fixed point to a user visible floating point UTC number.

raveslave commented 2 years ago

checking in on this one, I find myself wanting to combine the saleae and joulescope plots all the time

my hack has been to measure a less exact current using say an INA223 to get a voltage output to connect into an analog channel on the saleae.

a dream feature on a future hardware version of joulescope would be either

mliberty1 commented 2 years ago

Hi @raveslave, and thanks for the feedback! We are working on the next-generation Joulescope now. The current design in progress has 4 digital general-purpose inputs and 2 digital general-purpose outputs. This will give you a couple more inputs to help with synchronization.

For the analog output feature, what output sampling rate would you need for your application? Would 0 to 3.3V output with zero biased to 1.65V be good enough? Our existing microcontroller does have a 12-bit, 1 Msps DAC that we could potentially use for this feature. I am not sure if it has enough processing remaining for the full rate.

I definitely understand your desire to have more analog inputs. I do think that there could be a market for a DAQ focused on the low-power space. Unfortunately, the current supply chain problems make it very difficult to consider any other new products at this time.

raveslave commented 2 years ago

hi, looking forward to the next version! great with some added i/o, a nice feature would be if the inputs can double as UART and I2C in case you want to trig on a specific i2c event or just get a timestamped UART log that can be synced to the current graphs (log could just be a socket to terminal for those who run the py bridge)

I find myself wanting to measure at least 3-4 voltage rails in parallel, shouldn't att too much to the bom to allow this as an extra to the main current/voltage input!?

about the CV ouput, does it need to go via the ARM or could you track & scale it in the analog domain? I would make it 10V range so that when using a scope or saleae (50 MS/s btw) you can simply do a "1:1" conversion if the range is 1A in joulescope, that would be 10V on the scope.

mliberty1 commented 2 years ago

We plan to support UART RX with timestamped messages for the next-generation Joulescope, the JS220. Depending upon how things go, this feature might be added after the initial launch, but we are designing for it! Timestamped I2C and SPI decode would be good to add, but those features would be lower priority than UART. The JS220 will have a much bigger FPGA than the JS110, so we will have more options open.

The JS220 will have dedicated V+ and V- inputs. You could potentially use an external analog multiplexer to sample multiple rails. I am not sure what exists off-the-shelf. Right now, adding any additional hardware is almost a non-starter due to the component supply shortage. I don't see any easy way to expose 3-4 more inputs with the existing design.

The DAC is on the USB host side, so the signal would be referenced to USB ground. We could do 0.1V / A. No promises on being able to add this feature, but I will investigate. The deciding factors will be cost and component availability. For many applications a simple digital signal output triggered on an analog threshold may be enough. The JS220 will have this feature.

raveslave commented 2 years ago

great features. hoping that the cpu dac might happen, it's a good extra.

just got an idea to an alternative "saleae mixed measurements" that might work on current hardware too! utilizing one of the digital outputs, the joulescope could output a light-weight, streaming uart-packet that is converted using a logic2-extension + High Level Analyzer to show the result as a text-string in their UI

mliberty1 commented 2 years ago

The JS220 will also have the ability to output UART TX (N81), too. Sending short text strings should work great with the existing Saleae Logic UART analyzer.

The JS110 general-purpose outputs are both controlled directly by the host software, so the timing is not great. USB variability is measured in milliseconds, so I don't see a way to get any reasonable data rates. The easiest path to UART RX/TX support is to upgrade to the JS220.