Merge 2024 04 developments

[Short-bus]

circuitpyhon/code.py

• Breaking change! You must update src/pilomar.py to match this version.
• Runs on CircuitPython 7.2 and 8.2.
  • Not yet running on CircuitPython 9.x
  • Current recommendation is CircuitPython 8.2
• General code refactoring to save memory.
  • Removed unused imports.
  • Removed unused methods from some classes.
  • LED task list simplified, removed unused states.
• Common UART channel config for 7.2 and 8.2 CircuitPython environments.
• Smoother motor movement if using microstepping.
  • CircuitPython clock is only accurate to whole seconds which caused movements to bunch up.
  • New solution simulates more accurate clock which spreads movement evenly.
• Trajectory now works on STEP POSITION rather than ANGLE.
  • This is a breaking change. src/pilomar.py must also support this.
• Slewmicrostepping supported.
  • You can have different microstepping settings for OBSERVATION vs GOTO/HOME moves.
  • This allows the telescope to move into position more quickly.
  • This requires the latest src/pilomar.py to support it.
• Backwards compatibility with earlier message formats removed because this is already a breaking change.
• RPi can send direct "pin" commands to turn microcontroller GPIO pins on/off.
  • This is a development and debugging feature.
  • The latest version of src/pilomar.py allows manually entered commands to trigger this.

data/ngc.json

• The NGC targets have been expanded to include the later IC targets too.
  • You need to delete any existing /data/ngc.pkl cache file if you want to use the new catalog.

src/pilomar.py

• General refactoring continues.
• Initial support for Raspberry Pi 5.
• Pilomar now runs on RPi3B, RPi4B and RPi5.
  • RPi3B must use the original BUSTER 32bit O/S. Some new features are not available on BUSTER builds.
  • RPi4B can still use the original BUSTER 32bit O/S. Or you can upgrade to BOOKWORM 64bit (significant performance improvement).
  • RPi5 must use the latest BOOKWORM 64bit O/S.
• Reduced repeated log messages during observation loops.
• New FolderHandler class replaces old FolderList.
• Observation related directory structure is now only built as needed.
  • Previously all possible directories were built when target was chosen.
• SessionHistory now uses a new sessionlist class.
  • Now stored in a JSON file. (Previous SessionHistory from older versions will be lost when you update to this)
  • The change is introduced to support planned future features.
• FastCapture and FullCapture modes introduced.
  • FullCapture is the traditional solution. Images are fully processed as soon as they are captured. Eg: .DNG raw data is extracted and saved before capturing the next image.
  • FastCapture is an option in the Parameter file. Images are captured as quickly as possible, final processing must be done separately when the observation has completed. Use the Process Image Files option on the Camera Tools Menu to complete the processing at the end of the observation.
• Aurora target type added.
  • Uses new FastCapture mode to gather images quickly with minimum processing during the observation.
  • Produces Keogram of the aurora activity at the end of the observation.
• When simulating images, you can now simulate light pollution, image noise and electrical field noise.
  • This is to aid with development/testing. Controlled via Parameter file.
• Dashboard displays use softer colour scheme (background now very dark gray, colors can still be changed in parameters).
• Startup warnings if OptimiseMoves or SlewMotor parameters enabled.
• Improved error detection when trying to find USB storage for images.
  • Program now issues more clear warning and lists common causes of problems.
  • Less likely to terminate.
• Camera Tools menu lets you select 16mm or 50mm lens.
  • Makes it simpler to set the parameters correctly for each lens.
  • You still need to restart the program to propagate the new settings fully.
• Timelapse feature added.
  • You can now define a delay between each image if you want to capture a timelapse sequence.
  • For example, creating a timelapse of a moonrise.
• astrolens, astrosensor and astrocamera classes moved out to separate include file.
  • Easier to exchange for alternative camera classes in the future.
• You can now send a manually typed command to the microcontroller.
  • Development/debug feature.
  • Do this from the MONITOR COMMUNICATIONS option, press the 'c' key to enter a manual command.
• Microcontroller communication thread now closes automatically if main thread fails.
• Horizon parameter added.
  • Observations and trajectories will not allow observations to continue below the 'horizon'.
  • This means that the MinAngle for the altitude motor can now be below 0degrees if you need it.
  • -ve degrees allows you to park the camera below horizontal - for example if you want to HOME the lens up against a cover or shade.
• Warning if Comet data from Minor Planet Centre is getting old.
  • Comet data should be refreshed periodically as orbit data and new comets are regularly updated.
  • Refresh is not yet automatic, you have to delete the old data file when you have an internet connection available to download a fresh copy.
• Exercise Motor options added to Motor Tools menu.
  • This automatically moves the motors through their entire range of movement. For build/dev/debug support.
• You can now adjust the exposure time DURING an observation.
  • Use the "+" and "-" keys to increase or decrease the exposure time.
  • "+" will DOUBLE the exposure time. Eg from 10 seconds to 20 seconds.
  • "-" will HALVE the exposure time. Eg from 10 seconds to 5 seconds.
• Preview image includes a RA/DEC direction compass.
  • You can see which in direction RA and DEC increase/decrease on the diagram.
• GPIO handling moved into separate pilomargpio.py library.
  • Pilomar can select between GPIO and GPIOD libraries depending upon the O/S.
  • RPi3 & RPi4 use the traditional GPIO library.
  • RPi5 uses the alternative GPIOD library.
• EnableCleanup/DisableCleanup processing improved.
  • Previous version did not separate raspistill and libcamera requirements properly.
• Basic FITS files can now be generated.
  • Some image processing software wants images in .FITS format rather than .jpg or .dng
  • On BOOKWORM O/S builds there is now an option to export as .fits
  • BUSTER builds don't support this.
• You can now decide which types of images are captured from the Camera Tools Menu instead of having to edit the parameter file.
  • JPG, DNG or FITS (See src/pilomarfits.py below for more detail)
  • pilomar will select the appropriate camera driver for the O/S and image types.
• If you edit the Parameter file from the menu, you must now restart the program for it to take effect.
  • The edit parameters option is a convenience function, but does not yet fully re-apply all parameter changes dynamically.
  • Earlier versions allowed you to reload the file during operation, but they did not safely update all instances of the new values.
  • New version will not allow any motor moves to be made until program is restarted.
• Improved support for microstepping.
  • Microstepping makes the system smoother and quieter in operation.
  • New 'Slew' mode allows larger step moves for HOME/GOTO movements. (SlewMotor parameter)
  • Options now on the motor tools menu to turn on/off and to select common/recommended microstepping configurations.
  • Program must be restarted after changing motor configurations to make sure changes propogate fully.

src/celestrak.py

• Improvements to data caching and refreshing logic.

src/pilomarcamera.py

• New library to support different camera classes.

src/pilomargpio.py

• New include file. GPIO handling is moved into this file. There are new classes for the classic GPIO handler and also classes for the newer GPIOD handler. This is to enable RPi 5 support.

src/pilomarimage.py

• Some refactoring.
• Initial EXIF tag extract available. More development to come.
• Logging functionality reworked across all modules.
• Added pilomarkeogram class. Related to mapping aurora activity.
• Removed old keograph class. Not used.

src/pilomarlogfile.py

• Nolonger flush log file writes immediately to storage. This was hitting the SD card very hard.
  • FastFlush parameter can be enabled to perform immediate flush of write buffers IF required.

src/textcolor.py

• Updated to latest version. (0.0.6)

src/pilomarfits.py

• New utility to save JPG & FITS format image files.
  • Only works on BOOKWORM operating system builds (it requires picamera2 library).
  • This feature is early in development, it probably needs improvement, but good to get some feedback if anyone uses it.
  • If saving in JPG & FITS at the same time, the .jpg format is a very simple debayered copy of the raw sensor data. If you want higher quality .jpg files I recommend not saving .FITS at the same time.

docs/Pilomar manual 2024-07-04.pdf

• Latest draft of manual for pi-lomar to match this version. Still a work in progress.