Closed vnp85 closed 2 months ago
The most likely explanation is that you have incorrectly set the "pixel size" in observation details. Can you double check?
This is my log, the pixel size looks correct.
` 13:41:41.234 [INFO] Output directory set to D:\2024-06-24\Sun-He-rp 13:41:41.236 [INFO] File 2024-06-24-0749_3-Sun-He-rp.ser 13:41:41.236 [INFO] Found Sharpcap metadata. Camera ZWO ASI678MM Binning 1 13:42:21.765 [INFO] Dimensions of raw image determined : 3696x5309 13:42:21.808 [INFO] Processing will require approximatively 673.67MB of disk space 13:42:21.978 [INFO] Detected spectral line Iron (Fe I) with binning 1 (for pixel size 2.00µm) 13:42:21.984 [INFO] Starting reconstruction... 13:42:21.984 [INFO] Distortion polynomial ax3 + bx2 + cx + d = 0
`
PS: it did work well with this average, containing a prominent sodium line
Would it be possible for you to share the SER file?
Yes, give me a sec.
If the pb is that JSol'Ex confuses the Fe I line with Sodium D2, you should be able to select one or the other explicitly.
Letting the user edit the proposed number and method of extraction, could make it possible to inspect other lines as well.
Helium D3 (method: subtraction) [pixel distance] Some line (method: as is) [pixel distance]
Some line: like hydrogen epsilon, near CaH
So this is the ser with the mis-detection. One can actually see, in the flash spectrum, the magnificent proms visible today, so there's the Helium line, no doubt. (The hosting server is a raspberry, a bit slow on the bandwidth)
(~10 GB) http://narnia.go.ro/seagate2tb/2024-06-24/2024-06-24-0749_3-Sun-He-rp.ser
Here's the prom, frame extracted via AS4 and gamma applied
Thanks, I'll take a look. 10GB SER file wow :) Regarding editing pixel distance, in this case it is easier to use scripts. It doesn't really make sense to add a parameter here since it is designed to work automatically. If it fails, then it's a bug that I should fix (or misconfiguration). You actually have one sample script which does Helium line extraction that you can tweak when you open the ImageMath mode (see also https://melix.github.io/astro4j/latest/en/jsolex.html#imagemath).
Thank you, I will take a look at the ImageMath helium alternative. The hydrogen epsilon isn't a big deal, I can just manually type a pixel shift value.
Off topic: do you know of a script that, given an image input (a snapshot from Sharpcap) can come back with wavelengths and lines seen (don't want to reinvent warm water)? Ideally with no parameters but reference data like snapshots around known spectral lines, like the Fraunhofers, either the Sodium or the Magnesium should be good starting points...
Off topic: do you know of a script that, given an image input (a snapshot from Sharpcap) can come back with wavelengths and lines seen (don't want to reinvent warm water)? Ideally with no parameters but reference data like snapshots around known spectral lines, like the Fraunhofers, either the Sodium or the Magnesium should be good starting points...
Yes, you can use INTI Map for this. It's still experimental AFAIK and I've got varying results on it.
Off: Regarding the spectral line question, and see also the Helium extraction question -- I identified the line I was looking for as the He I at 447.15 nm, exactly as planned my session, but wasn't sure about the line. However, the result is ugly -- even though intellectually cool.
Ah right, but JSol'Ex automatic processing is only available for the D3 line. You can do exactly what you want with the scripts though. In particular look at the find_shift
function.
I'll look into it.
I hope you find the bug around the D3 line (I think it's going to be a pattern thing, though my Java knowledge is more than rusty). Visually, having seen this on my screen before switching to slit view, was mesmerizing.
Is this done with a Sol'Ex or another SHG? Seems that the resolution if fairly high. From what I can tell, your crop window does not contain the Fe I line, which explains that JSol'Ex mismatched.
Very same Sol'Ex and 678MM with 2um pixels, different crop. I double checked.
I have a vertically full size still, a larger video that does contain a Sodium D line, and then several that don't contain the Sodium D line.
The one on the left, with the Sodium line works, and extracts Helium, the one on the right doesn't, albeit it is a crop of it.
The left one's logs
`12:29:20.685 [INFO] Output directory set to D:\2024-06-24\Sun-He 12:29:20.686 [INFO] File 2024-06-24-0741_6-Sun-He.ser 12:29:20.686 [INFO] Found Sharpcap metadata. Camera ZWO ASI678MM Binning 1 12:30:01.463 [INFO] Dimensions of raw image determined : 3696x2925 12:30:01.476 [INFO] Processing will require approximatively 371.16MB of disk space 12:30:01.816 [INFO] Detected spectral line Sodium (D2) with binning 1 (for pixel size 2.00µm) 12:30:01.845 [INFO] Starting reconstruction... 12:30:01.845 [INFO] Distortion polynomial ax3 + bx2 + cx + d = 0
The right one's
` 13:41:41.234 [INFO] Output directory set to D:\2024-06-24\Sun-He-rp 13:41:41.236 [INFO] File 2024-06-24-0749_3-Sun-He-rp.ser 13:41:41.236 [INFO] Found Sharpcap metadata. Camera ZWO ASI678MM Binning 1 13:42:21.765 [INFO] Dimensions of raw image determined : 3696x5309 13:42:21.808 [INFO] Processing will require approximatively 673.67MB of disk space 13:42:21.978 [INFO] Detected spectral line Iron (Fe I) with binning 1 (for pixel size 2.00µm) 13:42:21.984 [INFO] Starting reconstruction... 13:42:21.984 [INFO] Distortion polynomial ax3 + bx2 + cx + d = 0
My spectral resolution is high, I can easily see the rugged inside, the splitting of the Calcium lines, for example -- random frame of the Sol'Ex in Ca H
and right next to it, hydrogen epsilon
So, is that particular line's presence a must? Marked on the image
Yes, as I said, auto-detection of the He D3 line will only work if either the Fe I or So D2 lines is visible in the crop window. If it's not, JSol'Ex may confuse other lines with the Fe1/D2 lines. This doesn't prevent you from using a different reference line, but in this case you have to go through scripting.
Thanks for clarifying.
The problem, perhaps for others as well, will be that the small pixels increase the vertical distance between the line candidates, and the data penalty, being additional area, is a square. I have 5 pixels (of 2 μm) where others have 4 (of 2.4 μm).
The problem isn't much the distance between the lines, it's having a reference to compare with. However, you may want to use binning to reduce the size of your SER files. Using bin1 works if you have excellent observing conditions.
The computational penalty for having to include a pixel-distant spectral line is the readout speed / framerate in particular, not the video size per se. If there is curvature in the line, it makes things much worse.
Binning may or may not be a solution, as spectrally I feel like I have wonderful resolution.
(I just checked the difference between CaH on band and plus-minus 3. Binning is out of question :D)
Hi Cèdric: I've been out of town a week. Now, when I use V2.5.1, the autodetection for He_d3 with a Sodium_d2 line is broken, it only produces the d2 line result. This autodetection function worked nicely in V2.4.9. Did a procedure change? Did the code change?
Steve
Here's part of the (new) problem. Even though I have a binning of 1 selected in the observation details, and both the Sodium d2 line and the He d3 line are in the frame, the run log shows this:
"Detected spectral line H-alpha with binning 2 (for pixel size 2.00µm)"
There must have been a change in the code which has somehow hard-coded bin 2 when Autodetect is used for the wavelength.
I have also experienced problems with the 2.5.1 release and Sodium, (namely getting stuck with the sodium image processing), I think that issue is a bit different than what I exposed here.
In the V2.4.9 version, similar frames produced this proper comment in the log:
11:11:01.420 [INFO] Detected spectral line Sodium (D2) with binning 1 (for pixel size 2.00µm)
I have verified that manual pixel offset does produce a helium image in my new SER files. Nothing has changed on my end since V2.4.9.
There must have been a change in the code which has somehow hard-coded bin 2 when Autodetect is used for the wavelength.
There is no such thing. You have to be aware that spectral line detection is a sensitive algorithm, which works reasonably well, but can fail. Therefore, unless the same file worked in 2.4.9 and fails in 2.5.1, this cannot be considered a regression. I personally regularly experience failures in auto-detection, and experienced that even in 2.4.9. However, should the detection fail, you always have the ability to set explicitly the line that serves as reference in the cropping windows (e.g Sodium D2 or Iron Fe I).
The binning detection only triggers if there's no .txt metadata file alongside the SER file. Is this the case?
The binning detection only triggers if there's no .txt metadata file alongside the SER file. Is this the case?
Is JSol'Ex able to detect the cameraSettings.txt even when the names are a bit off? Here's an example from my archive, it happens when I record with the SHG: opening the record window some 30 seconds before the actual recording places the first frame of the ser file.
Is JSol'Ex able to detect the cameraSettings.txt even when the names are a bit off?
No, the file names must match.
Okay - I can verify that V2.5.1 does properly autodetect a SER file that was previously correctly autodetected by V2.4.9. So my new files from today might be a bit off for a sensitive algorithm - I was fighting with high cirrus clouds in the early part of the morning. The d2 and d3 lines were clearly there, but slight cirrus might have caused an issue.
I do not understand why, if I explicitly declared Bin 1 in the observation details, the program can ignore this in the autodetect when autodetect fails. The camera was set to bin 1. I have metadata in the metadata tab, but I'm not sure of the file you are referring to for metadata - I only see the run log in the window manager. Is there a location for metadata info?
I do not understand why, if I explicitly declared Bin 1 in the observation details, the program can ignore this in the autodetect when autodetect fails. The camera was set to bin 1. I have metadata in the metadata tab, but I'm not sure of the file you are referring to for metadata - I only see the run log in the window manager. Is there a location for metadata info?
Right let me clarify. There's the acquisition software metadata file, which is the .txt
file which lives alongside the SER file, and is written typically by SharpCap. If this file is found, then the camera name and binning are read from this file automatically, and assumed correct. Therefore it doesn't matter what you set in the observation details, it's this metadata file which tells us what to use.
The metadata tab is something specific to JSol'Ex, and shows the different operations which were executed on an image, the pixel shift, etc...
I doublechecked it, kind of a false alarm. It's when it's WinJupos filenames checked in the sharpcap options, AND also interference from my folder-watching renamer, as Sharpcap intends to rename the settingsTxt when it's done recording. Thus, unrealted (and in fact a bug in my ersatz-obsi software stack). As a side effect, in my case it could be that it is this that hinders the binning detection ineffective, although in the opening dialog of JSol'Ex, I manually set the binning and pixel size to the right values.
Hi Cedric:
So - the answer to your acq metadata question is yes - the acq metadata .txt file is always present alongside the ser file in the same directory, for both the cases that fail autodetect and those which pass. My capture software is Firecapture. Here is the file text for a failing autodetect which mistakenly indicates bin2:
Camera=QHY5III678M Filter=L Profile=Sun Filename=Sun_073954.ser Date=062724 Date(UT)=062724 Start=073943.076 Mid=073954.757 End=074006.439 Start(UT)=143943.076 Mid(UT)=143954.757 End(UT)=144006.439 Duration=23.363s Date_format=MMddyy Time_format=HHmmss LT=UT -7h Frames captured=2988 FPS (avg.)=127 File type=SER Binning=1x1 Data=Mono ROI=3472x340 ROI(Offset)=200x484 Shutter=4.900ms Gain=38 (15%) Offset=0 Brightness=0 Contrast=5 HighSpeed=off USBTraffic=0 Histogramm(min)=0 Histogramm(max)=5 Histogramm=0% Noise(avg.deviation)=n/a Limit=10000 Frames
---------- ROI offset positions ---------- Frame 1: 200x484
Thanks, but as I said, it's not because you have the metadata file that autodetection will succeed. IF there's a metadata file, then the binning that it contains is deemed accurate, but that's pretty much it.
Did you test this with Firecapture? The Firecapture metadata is declaring bin 1 as the string "1x1". Not sure if that's what Shartpcap says for Bin 1. I thought jSolex was using the metadata. True?
It's declaring bin2 even though the metadata says bin 1x1.
Yes I tested this with Firecapture. It's fairly easy to tell if the metadata file was found, the logs will contain an entry such as:
Found Firecapture metadata. Camera ZWO ASI178MM Binning 1
Here's a logfile from a successful autodetect session. It does not contain the entry you mention, even though the acq txt file is there, beside the ser file on the same level in the directory.
14:23:35.275 [INFO] Output directory set to /Volumes/T7/Firecapture/Sun/061824 14:23:35.275 [INFO] File Sun_065905copy.ser 14:24:39.048 [INFO] Dimensions of raw image determined : 3500x2973 14:24:39.050 [INFO] Processing will require approximatively 535.87MB of disk space 14:24:39.061 [INFO] Starting reconstruction... 14:24:39.061 [INFO] Distortion polynomial ax3 + bx2 + cx + d = 0
what's the file name of your .txt file? It should be Sun_065905copy.txt
It is. And in no other of my autodetect or any other sessions, is a mention of the metadata file found in the logfile. Here's the directory structure...
Text from a different day, different successful autodetect, metadata file present but no mention of it in the log:
10:06:53.991 [INFO] Output directory set to /Volumes/T7/Firecapture/Sun/061724 10:06:53.991 [INFO] File Sun_071422.ser 10:07:54.932 [INFO] Dimensions of raw image determined : 3424x2934 10:07:54.934 [INFO] Processing will require approximatively 517.35MB of disk space 10:07:55.042 [INFO] Detected spectral line Sodium (D2) with binning 1 (for pixel size 2.00µm) 10:07:55.042 [INFO] Starting reconstruction... 10:07:55.042 [INFO] Distortion polynomial ax3 + bx2 + cx + d = 0
I think I have found the bug with parsing Firecapture metadata. Can you try the development version?
Cèdric: I'll try it tonight and report back. Thanks for checking it out.
Cédric:
Results were excellent with the parsing fix in V2.5.2. Two things happened.
Here's the log:
Output directory set to /Volumes/T7/Firecapture/Sun/062724 File Sun_073848.ser Found Firecapture metadata. Camera QHY5III678M Binning 1 Date 2024-06-27T14:38:37.448 SER file contains 2904 frames Color mode : MONO (2 bytes per pixel, depth = 16 bits) Width: 3472, height: 340 Solar parameters Carrington rotation = 2285, B0 = 2.51°, L0 = 30.39°, P = -4.02° Computing average image and detecting limb Dimensions of raw image determined : 3472x2903 Processing will require approximatively 519.06MB of disk space Detected spectral line Sodium (D2) with binning 1 (for pixel size 2.00µm) Starting reconstruction... Distortion polynomial ax3 + bx2 + cx + d = 0
Here's the info in the jSolEx Metadata tab:
Image
Pixel Shift = -249.90 pixels Transformation history
Ellipse parameters C(x,y) = ax² + bxy + cy² + dx + ey + f = 0
Carrington rotation = 2285, B0 = 2.51°, L0 = 30.39°, P = -4.02° PixelShiftRange = PixelShiftRange[minPixelShift=-280.0, maxPixelShift=20.0, step=30.0] Reference coordinates:OFFSET_X[490.32778407505384] OFFSET_Y[-91.00803874030976] MARKER[0.0]
With this encouraging result - I will try some Bin2 tomorrow on the Helium and the Ca-H lines, if the clouds cooperate. Observing in Phoenix Arizona in the summer must be done before 0900 local time, or it gets too hot outside.
Merci Cèdric!
Followup:
As of this morning with V2.5.2dev, Bin2 mode works great on He_d3 and Ca-H, where I do not need the full camera resolution and large file size. jSolEx picks up the metadata info and autopopulates the camera fields in the Observation Details form - I don't remember it doing this before. Significant bug fix - thanks.
First of all, thank you for your software.
Bug description I open a ser recording of a spectroheliogram around the helium D3 line. Quick mode misidentifies the spectral line, and the resulting Helium D3 extraction is just noise.
Perhaps make this into a feature request Make the number, next to the Helium D3 editable.
Reproduction Ver. 2.5.0 Use a recording that has no Sodium D lines in it, and the iron etc. in the vicinity are slightly obscure. Run quick mode. If the Helium D3 option shows up, chances are, it misses the line. In the uploaded image it is at -66 pixels from the detected spectral line, while JSol'Ex proposes -142.
Expected: the Helium D3 number to be around -66 in this case Observed: the Helium D3 extraction, if available at all, is -142 or so
I uploaded the average png, the helium line is around the middle of the image, two absorption lines below, one above it.