sn4k3 / UVtools

MSLA/DLP, file analysis, calibration, repair, conversion and manipulation
GNU Affero General Public License v3.0
1.17k stars 103 forks source link

Report on use with .pwmo files #149

Closed Nerivec closed 3 years ago

Nerivec commented 3 years ago

I have been using UVTools with the .pwmo format for a few days and thought I'd report the issues I have encountered and suggestions I can make so far.

File values

Header > PixelSize

For Photon Mono, this should report 51, the um size of pixels, however, the field is never set (stays at default 0.04725). Setting it manually (using HxD) fixes the incorrectly reported X/Y size in View>Machine information (in Photon Workshop).

Header > ResinType

This one appears to actually be the Currency as per the setting available in Photon Workshop ($ 24 00 or ¥ C2 A5).

Header > Offset1

This seems to be what Photon Workshop uses to display total print time (albeit poorly calculated by all slicers...).

LayerData > LayerPositionZ

UVTools sets this as actual Z position for a given layer. All other softwares (Photon Workshop, Chitubox, Lychee) set this as layer thickness. I have not tried if that would actually influence the printing process (if it does, it would most certainly result in failure). It might be better to follow Photon Workshop behavior here.


Example with a converted sl1>pwmo, edited with HxD as mentioned above: pwmo

Improvements

Rounding

Since a lot of values are in micron, I think a greater value than the current 2 digits would be good for rounding.

Adapt Z per layer based on resin amount being exposed?

Looking at the TEST and R_E_R_F files supplied by Anycubic, they seem to be using an algorithm to override Z Lift Distance and Z Lift Speed per layer, based on the amount of non-black pixels. See image below. I have yet to dive into it, but I'm sure we can figure out the formula from the Anycubic files and go from there, so UVTool can do this automatically with a simple function call. I would think that in some cases, it might greatly reduce the print time, while having no impact on print quality, unless someone more experienced can point to something bad in doing this...

per-layer-override

Here are the files, directly from the Anycubic USB drive that came with the Photon Mono printer: TEST_and_R_E_R_F.zip

Nerivec commented 3 years ago

I forgot to mention this, I'm not sure if you have noticed, but Photon Workshop uses a folder in AppData\Local\Temp\ANYCUBIC to store slicing info. It makes quite a mess too (it stores all layers in there, after a few slicings, let's just say, the folder gets crowded fast...). There are 3 files worth checking out in all this (written on each slicing), here they are, with default settings everywhere, using the most recent Ameralabs test stl (like above):

machine.mac

[machine]
type=PhotonMono
suffix=pwmo
xres=1620
yres=2560
xyPixel=51.00
xsize=82.62
ysize=130.56
samples=8
xblank=2.62
yblank=0.56
zsize=165.00

print.info

[printinfo]
layer_count=305
volume=1.574
print_time=2959
cost=0.686
cost_currency=\xa5
individual_layer=false

slice.para

[autoConfig]
mode=1
prefer=0
Ranges\zHeightMin=0.400
Ranges\zHeightMax=6.000
Ranges\zHeightSteps=1000
Ranges\expTimeMin=3.000
Ranges\expTimeMax=12.000
Ranges\expTimeSteps=1000
Ranges\zLiftSpeedMin=3.000
Ranges\zLiftSpeedMax=10.000
Ranges\zLiftSpeedSteps=1000
Ranges\liquidHeight=30.000
Ranges\LiftCompenHeight=2.000

[resinProperty]
type=Basic
price=218.00
currsym=\xa5
volume=500.00

[resinParameter]
minlayerHeight=0.000
maxlayerHeight=10.000
minoffTime=0.00
maxoffTime=3600.00
minbottLayers=0
maxbottLayers=1000
layerHeight=0.050
normalExpTime=2.00
offTime=0.50
bottExpTime=40.00
bottLayers=6
zLiftHeight=6.000
zLiftSpeed=4.000
zRetractSpeed=6.000
antiCount=1
sn4k3 commented 3 years ago

Header > PixelSize

Yes i had it static to a value which is also wrong because was in mm and this is um. I didn't care much because that value is information only, not used by printer and slicer should compute value based on display width/height and show that as informative only. Still i will fix it and encode proper values for each printer, and also define Displaywidth and height as that is usefull for some tools inside UVtools. While UVtools use PixelSize on some tools, it is still computed and not rely on any set info on the file format.

Header > ResinType & Offset1

Thanks for the finding, i will implement as such

image

LayerData > LayerPositionZ

Chitubox doesn't show this value but they use it internally, also it's all a matter of naming format, for me i will call it LayerPositionZ as that mean the real position in Z where the layer will cure at, calling it layer thickness is wrong since that is the layer height eg (0.05mm). Photonworkshop shows as Z Layer height which makes kinda of sense but still not the best name to describe the position of a coordinate. Every slicer will call their fields diferent names, UVtools will too, and that doesn't matter. PS: LayerHeight = LayerPositionZ - PreviousLayerPositionZ

Rounding

Which values you find in microns? i only see the pixelsize... Also mostly microns dont require decimals most of the time they are integers. About the other values i round everything to 2 or 3 places because slicers didn't care about them and they put the fields with numbers with precision errors eg: 9 decimals places, which is ugly and can lead to errors if firmware don't round them internaly. There is no need to have more than two decimals, for example exposure time why do you want to use a 3rd digit? About layer height and lifts i also force to two at purpose, some people are slicing 0.015 layer height and then have quality problems, because cheap machines can only have a precision of 0.01mm, going beyond that the stepper start to put your Z at wrong minimal heights and inconsistent layers start to show, same for Lift height, using 1 decimal is more than good, i allow two, three is overkill /lead to problems / useless to have. Not to mention that on GUI showing lot of decimal plates are ugly ^^ UVtools is armored against that precision errors from floats math, most of properties have rounding on setters, getters and GUI itself have rouding on formatstring to 2, so it's really a enforce! I have seen chitubox files come from slicer with layer position z something as 0.089900009mm, if firmware dont round that Z start to variate positions overtime. The good thing is UVtools sanitize all that (If you save the file).

Adapt Z per layer based on resin amount being exposed?

That is a good feature to have, but very few printers support that, and the problem is you can have a huge mass but then you also have tiny masses that require more time, this can be better suited with grey out pixels other than using exposures which you cant do by zone, even by layer almost no printer support that.

I have seen some photonworkshop files with random values on that fields, i don't know what they are doing but thier slicer is a mess, it don't matter if they customize that value or not, because the "Enabled" radio is OFF which mean it will not use that overrides, Most of the values you see there are the Bottom values which will show for bottom layers, and the Normal values which show after the bottoms. I don't know if anyone test that overrides but i hear that can't do nothing even if enabled, not sure now with most recent firmware. You can test it out by using UVtools tool - Edit print parameter and modify per a layer range. Never change values on that PhotonWorkShop dialog and resave the file, never but never reslice a sliced file.

image

As you can see you are modifing a layer range which is bottoms or not, and can't select your own range

I forgot to mention this, I'm not sure if you have noticed, but Photon Workshop uses a folder in AppData\Local\Temp\ANYCUBIC to store slicing info. It makes quite a mess too (it stores all layers in there, after a few slicings, let's just say, the folder gets crowded fast...). There are 3 files worth checking out in all this (written on each slicing), here they are, with default settings everywhere, using the most recent Ameralabs test stl (like above):

UVtools don't look nor care about slicer generation / dumps / saves. It only reads final files the user send into. But good to know where it does the temp generation, every slicer must do it, or else you would require a ton of ram to keep all images while processing. That dump is only to debug, the actual sincer don't reuse that as they will regen that information everytime you slice.

Thanks for your time debugging and bulding this information, i will send the improvements on next patch

Nerivec commented 3 years ago

Yeah, Photon computes the display size from the pixel size rather than the other way around and it doesn't seem to store them in the pwmo file, not that I could find anyway. It all is just informative anyway, like you said.

About Layer Z Position, it wasn't much about the naming convention that I mentioned it, and I agree, it makes more sense to have that variable show the Z position, rather than just copying the layer thickness param in there for every single layer, but Photon has it setting that way (i.e. no matter what layer you are looking at, for a Photon sliced file, Z Layer Height = Layer Thickness). I suppose this could have been originally designed as a parameter to allow layer height override per layer (for example, do detailed parts and things like straight parts with different precisions)... Either way, not sure if it actually matters to the printer... but if it does, it could create issues, for example, using the screenshot above, with Layer Thickness at 0.02 but Z Layer Height at actual Z position, here 15.18mm (rather than 0.02 as a Photon sliced file would normally have), if the printer reads that as Layer Thickness and tries to do a layer at 15mm precision, well... :boom: Have you tested that it is not the case?

About rounding, I meant using "micron" in the 3D printing world in general. About the variables themselves, some don't need the 3 digits precision for sure, while some would benefit from it, like layer height, considering the small values recent printers can reach; but true, if some printers don't support it, it could cause issues... I guess it could come in the form of an option in UVTools settings, something used application-wide (one for mm, one for sec)? (I guess Cost would be more accurate for smaller models too if we were to get down to the finer details (for those that print in bulk :yum:), but I'm pretty sure Photon rounds it to 2 despite showing 3 digits)

I haven't tested the layer override personally. I also am not sure if the checkbox would prevent the printer from using the "per layer" settings instead of the global ones (it could just be used by Photon, while the printer always look at layer info...), that would require some testing too. I did not pay attention to that when I originally printed the Anycubic test file... ;-)

I only mentioned the Photon config files for future debugging purposes, it can be useful to find values in .pwmo files by testing different slicing settings and seeing how the config files actually store them. Note for Photon Workshop devs though, the app needs a "clear cache" button... or even better, an auto cleanup on close... lol

sn4k3 commented 3 years ago

doesn't seem to store them in the pwmo file

It does, you can look at UVtools sources to see the whole file format table or just go File -> Extract and see the configuration files. Anyway what UVtools shows on information table on GUI is the actual data read from file format. But start here if you want to learn more about the internal of format tables: https://github.com/sn4k3/UVtools/blob/master/UVtools.Core/FileFormats/PhotonWorkshopFile.cs#L79

designed as a parameter to allow layer height override

Layer height on layer data is virtual/computed, if you want higher layer height you need to increase Z so it cure above last layer higher. And if you change that layer you need to change all the next by that offset. So i really think they never planed such thing because problems and complexity, that require to be done before slicing and not otherwise as they do. What photonworkshop shows is computed, because internally on format they are using only position z just like chitubox does, in fact this formats are just similar copies of each others.

About rounding, I meant using "micron" in the 3D printing world in general.

That only make sense for very small values like layer height, pixel size, other than that is all numbers that start on more than 1mm and that dont make sense, to use big numbers in microns as 5mm would be 5000um (worst reading) . Every slicer is using layer height as mm so i do too.

About the variables themselves, some don't need the 3 digits precision for sure, while some would benefit from it, like layer height

Maybe... going lower than 0.01mm have no point, since many people report from 0.02 to 0.03mm they see no visual diference, but can make sense for 0.035 for example, still i don't have plans to allow it yet. I can only see problems coming from that and really, there are no good reason to have extra 0.00x precision on this kind of machines. 0.01mm is smaller than a pixel on a 4K LCD! which have about 0.035mm, you can't see 0.01mm without a good zoom. Same for exposures or any other value, tune times to a 3rd digit is a time/resin loss and will not give you better results than you can already get. In other end if people slice to the 3rd digit layer height, eg: 0.035mm uvtools will round that and invalidate model height, maybe i should put a check in...

(I guess Cost would be more accurate for smaller models too if we were to get down to the finer details (for those that print in bulk 😋), but I'm pretty sure Photon rounds it to 2 despite showing 3 digits)

Cost are always not accurate, none print is cheaper than 1cent too. Even if you spend 0.1ml of resin all the other work involved kill it all

I haven't tested the layer override personally.

Me either, i dont have any machine other than a Sonic Mini

sn4k3 commented 3 years ago

It seens LayerPositionZ is really the layer height on the format, leading to troubles, someone also reported on fb groups. Will fix it too

Nerivec commented 3 years ago

It does, you can look at UVtools sources to see the whole file format table or just go File

Yes, I meant the display width/height is not stored in .pwmo, only resolution X/Y and pixel size, that's why when pixel size is wrong, Photon Workshop reports a wrong display width/height; but again, they are just informative values, no big deal.

Every slicer is using layer height as mm so i do too.

Yeah 5000um would be pretty ugly. Like I said, I didn't mean using micron as variable type, just that the 3D printing world is using micron-level values (indifferent of um/mm).

I had not come across the information that some printers would not recognize 0.005 precision before. That's interesting actually, especially since PrusaSlicer is using 0.025 and 0.035 as system presets for SLA print settings. I had noticed that my sl1<>pwmo converted files reported rounded values for layer height, that's what made me take a closer look at the UVTools code in the first place (I went down the rabbit hole from there to see if I could track down the missing variables :yum:).


I just ran a test without vat for the layer override thing (with the Photon Mono firmware v0.1.5). Interesting results for sure... So first off, when editing layer parameters in UVTools, it doesn't flip the PerLayerOverride bool. I flipped it using HxD and tried both the flipped (1) and non-flipped (0) versions. The non-flipped version ignores the layer overrides completely and uses the global parameters instead (exactly as it should). The flipped one however, partially uses the layer overrides. I say "partially", because it only does so after the bottom layers (i.e. for this file, bottom layers remained at 45sec exposure despite the 15sec in layer overrides). After that though, it started using the layer overrides (both exposure and Z Lift Distance/Speed).

I did notice one extra thing, having the layer overrides on, forces the printer to re-adjust the remaining printing time every time the layer settings change from layer to layer, rendering the estimate completely worthless in this mode. It seems to compute using current layer settings and multiplying by remaining layer count, from what I could tell.

I attached the modified file, in case you want to take a look. TEST_copy.zip

Also worth mentioning, since we now know the override thing works, I noticed that converting a .sl1 to .pwmo automatically enables PerLayerOverride. I'm not sure the behavior is intended, since by default, it is not needed.

sn4k3 commented 3 years ago

PrusaSlicer is using 0.025 and 0.035 as system presets for SLA print settings

They are using trinamic drivers paired with good steppers. Not the case for every printer but even they should not use that. I'm not the only one who advice this.

So first off, when editing layer parameters in UVTools, it doesn't flip the PerLayerOverride bool.

Yeah i will enforce that to be always true as UVtools sanitize all layers and that will cause no bad. After save the file it will auto set that field on next patch. However if you convert a file the default is 1 as i set it like that.

I just ran a test without vat for the layer override thing (with the Photon Mono firmware v0.1.5). Interesting results for sure...

Thanks for testing that, a big step over for sure, that mean you can make use of dynamic layer height tool on next patch, by just start on normal layers

On your test file they are able to have a custom range, need to find the trick, maybe is the offset fields

sn4k3 commented 3 years ago

Here a sample each layer table: https://pastebin.com/raw/BvLVH00G

And other tables:

FileMark -> Mark: ANYCUBIC, Version: 1, AreaNum: 0, HeaderAddress: 48, Offset1: 0, PreviewAddress: 144, Offset2: 0, LayerDefinitionAddress: 75420, Offset3: 0, LayerImageAddress: 141840
Header -> Section: [Mark: HEADER, Length: 80], PixelSizeUm: 50, LayerHeight: 0.05, LayerExposureTime: 2, LightOffDelay: 9.17, BottomExposureSeconds: 40, BottomLayersCount: 6, LiftHeight: 8, LiftSpeed: 2, RetractSpeed: 3, Volume: 61.021114, AntiAliasing: 1, ResolutionX: 3840, ResolutionY: 2400, WeightG: 0, Price: 26.605206, PriceCurrencyDec: 42434, PerLayerOverride: 0, PrintTime: 23690, Offset1: 0, Offset2: 0
Preview -> Section: [Mark: PREVIEW, Length: 75292], Width: 224, Resolution: 42, Height: 168

Btw see what i mean by round error on slicing: image

That is just ugly for LiftHeight... I really hope they do some rounding on firmware

Nerivec commented 3 years ago

On your test file they are able to have a custom range, need to find the trick, maybe is the offset fields

For the bottom layers you means? I think the printer bypasses the bottom layers overrides altogether, they probably meant to only use layer overrides for normal layers (makes sense) but Photon Workshop shows all of them editable when they shouldn't.

Here is what I did exactly with the file, just in case:

Btw see what i mean by round error on slicing

Yeah, I know, floating rounding is always a problem...

Here a sample each layer table

I have noticed quite a few of the offsets receive random values, even from slicing the same STL twice in a row without any changes in-between... On the TEST and R_E_R_F original files both Offset1 and Offset2 for LAYERDEF's layers are always 0 though.

sn4k3 commented 3 years ago

Yes bottom and no bottom, they have a custom layer range set in... I found what offset1 is ^^ Pixel count in layer. And yes i count them 1 per 1 to make sure!

Nerivec commented 3 years ago

The custom layer range you can get from UVTools, using the Edit print parameters, if you set your range, change the values, that will add a new range for whatever layers you picked. I'm not sure I understand what you are looking for?

I found what offset1 is ^^ Pixel count in layer. And yes i count them 1 per 1 to make sure!

Number of white pixels, damn, that's a good one! Makes you wonder, in all this, what the printer actually uses... :yum:

Here, for future counting needs! lol (I've been comparing AA features of various slicers anticipating the release of the damn fix for the Mono, this little script proved very useful to quickly check stuff!)

import sys, os, argparse

# pillow
from PIL import Image

parser = argparse.ArgumentParser()

parser.add_argument('-f', dest='filepath', help='File to process.')

args = parser.parse_args()

try:
    with Image.open(args.filepath) as img:
        print('Resolution: {}x{}'.format(img.width, img.height))
        print('Pixels: {}'.format(img.width * img.height))
        print(img.getcolors())
except OSError:
    print('Invalid file.')
python pixel_info.py -f AmeraLabs_calibration_part_town_raft/layer000.png
Resolution: 1620x2560
Pixels: 4147200
[(4009014, 0), (138186, 255)]
sn4k3 commented 3 years ago

Ah yes forget, i actually see, PhotonWorkshop sets the range automatic based on layer changes... Btw how can i change layer tickness on the program? It dont let me use other than 0.05mm :(

Number of white pixels can be used to calculate used resin volume

sn4k3 commented 3 years ago

Fixed, reopen if you find anyother issue, you can keep commenting here

Nerivec commented 3 years ago

Btw how can i change layer tickness on the program? It dont let me use other than 0.05mm :(

Change layer thickness on Photon Workshop you mean? If you change the number it should just work... it does on my side...


v2.4.7 does not have the right encoding for the currency symbol, it is stored as Hex UTF-8 bytes.

$ = 24 00 00 00
¥ = C2 A5 00 00
€ = E2 82 AC 00
etc...

Actually, I checked, there is no internal sanitizing or any filter by PW, you can write anything in the field using this, for example 45 55 52 00 will show EUR, or with 24242424 you can make 4 dollar signs show up...

I also found something wrong with the Preview section while looking a bit deeper, it has 4 offsets at the end, which results in pixel errors when using "Extract". It seems, if there were offsets there before, they no longer are present, at least in the PWMO format.

One last thing to watch out with the pwmo file format, the new firmware for Mono (only available in beta version for now) that fixes the AA, the feature only works with Photon Workshop (current version), which means both Chitubox & Lychee save them wrong somehow, at least until their next respective update. See this short video where he talks about it. That might solve one of the mysteries of the unknown offsets, or maybe it's because of the NonZeroPixels value not being set at all by CB/LC...

pwmo_analysis.xlsx

PS: For future reference, as you can see in the spreadsheet, make sure to never trust section length nor address values while reading the files (don't use it to offset a read operation for example). While address values for PW-created files are OK (header, preview, layerdef, firstlayer), they are not for Chitubox & Lychee ones. Section length values aren't even right for PW-created files in some cases (the Preview one is off, guessing they made some tweaks to the format and didn't properly offset the value calculation).

sn4k3 commented 3 years ago

Change layer thickness on Photon Workshop you mean? If you change the number it should just work... it does on my side...

Dont work for me :(

v2.4.7 does not have the right encoding for the currency symbol, it is stored as Hex UTF-8 bytes.

Will fix that

I also found something wrong with the Preview section while looking a bit deeper, it has 4 offsets at the end, which results in pixel errors when using "Extract". It seems, if there were offsets there before, they no longer are present, at least in the PWMO format.

Yes i know that offsets are a mess, and to help more it is diferent on mac and window on PW sometime there is others dont...

One last thing to watch out with the pwmo file format, the new firmware for Mono (only available in beta version for now) that fixes the AA, the feature only works with Photon Workshop (current version)

UVtools encode/decode for AA files for PWxx are also broken, we never know for sure what are they using

Thanks for your datasheet, i will compare with UVtools. This PW file formats are a real mess!

EDIT 1: on your preview_resolution, that is resolution of the image in dpi.

Nerivec commented 3 years ago

EDIT 1: on your preview_resolution, that is resolution of the image in dpi.

I saw the comment in your photon file code, yes. But what formula are they using? DPI/PPI is quite messy, but Photoshop reports 72 for the thumbnail not 42. Unless you're changing the file a little before saving it on Extract? I didn't look into the encoding/decoding fn.

Also, you should pad the preview image data before converting it to an image so you don't get pixel errors (and to avoid possible issues software-wise). That is, until Anycubic fixes the issue... If you take the length of the preview data and compare it to the expected size (width height 2), you can get the number of missing pixels, which should be 16 (75264 - 75248), then you can just repeat the last available pixel for these before creating the image... In most cases, you won't see the difference since it's in the very corner, but it does avoid potential issues (not sure how C# behaves on that front, but you can see right now that it is populating these pixels with various values, guessing it's just from a default value that gets shifted by the encoding function, still...).

Good news though, v2.1.25.RC2, Anycubic fixed the mess PW was creating in the Temp folder with layer images... Either that or they relocated the files :smile: And the layer count now matches that of CB/LC.

sn4k3 commented 3 years ago

But what formula are they using? DPI/PPI is quite messy, but Photoshop reports 72 for the thumbnail not 42.

DPI depends on LCD physical size, DPI = resolution / lcd size * 25.4 It doesnt matter it's all informative values. 42 was set for Photon Zero (PW0) i belive new PWMx printers have larger lcd for interface then more DPI

Also, you should pad the preview image data before converting it to an image so you don't get pixel errors (and to avoid possible issues software-wise). That is, until Anycubic fixes the issue...

I know that about previews, and also i know about last green pixels noise, i never cared as that is almost unnoticable. But i can do that. Some use pads before other after i dont know, but i talked to lychee dev about that and i think they take another approach to this (I dont renember atm) In fact on UVtools i remove the preview validation because of this, it eat anything without complain

Good news though, v2.1.25.RC2, Anycubic fixed the mess PW was creating in the Temp folder with layer images... Either that or they relocated the files 😄 And the layer count now matches that of CB/LC.

Yeah, that last black layer was just useless and uvtools will remove it on layer fix anyway

Nerivec commented 3 years ago

DPI depends on LCD physical size, DPI = resolution / lcd size * 25.4

Oh, it's the DPI from the printer's point of view! Got it... Thanks.

I know that about previews

Well, I guess as long as C# handles the difference correctly, it's fine, don't really care about a few missing pixels that you can't even see on the printer's screen. And I might add, Photon Workshop also displays the bad pixels...