1x1 baseplate .ipj out of date in MDK

[AlecVercruysse]

Hello,

It seems that MDK/CAD/Assembly_Cube_Baseplate_1x1/ only contains the inventor project files that pertain to v0 of the baseplate, while the baseplate folder in CAD/ contains v2 of the baseplate. Is there a chance you can provide the inventor project files for v2?

Related to this issue, since there seem to be many copies of what seem to be the same file in every repository (e.g. the STL files pertaining to each module in the Applications), are there plans to consolidate and "future proof" the design files by symlinking (or similar) to the files in the CAD folders? Or do you choose not to in case there are backwards compatibility issues with new versions in the future?

[beniroquai]

Hey, thanks for pointing this out! I'll have a look and will upload the proper v2-Inventor Project files. Regarding the STLs: Very good point. Ideally I would like to have something like the openflexure buildsystem. A very nice way to distribute stuff is this online configurator. Unfortunately my skills to program webapps is very limited. The idea would be to have a database (e.g. CSV) with a file hierarchy. A programm packs the stuff per setup and you'Re done. Always up-to-date..

If you're willing to help here, please let me know ;-)

[beniroquai]

Regarding the v0 vs v2 conflict: We didn't change the baseplate. So this stays as is. Nevertheless, we will have a new puzzle-piece baseplate soon. But for now the files are good :)

[AlecVercruysse]

Thanks! This looks like an issue with how SolidWorks imported the ipj file, it looks correct on Autodesk Inventor, and converting it to a STEP file fixed all issues.

As for your idea regarding a similar online configurator to that of OpenFlexure, how do you envision this working with the modular design philosophy of UC2? One json/yaml file is created per module with config options, as well as the rules for which file to include for each respective choice? Some sort of yaml file in each application folder specifies which modules to include for the application, which parameters might be required, and which might be flexible?

The webapp could scan through each module as well as each application to build it's database, and a dropdown/link could specify whether or not the user was looking to print a series of modules or a specific application?

This would require building

• a webapp (hosted on github pages? so using JS only)
• A config file for each module in CAD/
• A config file for each application in APPLICATIONS/
• Removing all APPLICATIONS/*/STL/ directories (once it all works)

The workflow for submitting a new application would then be a pull request including just the additional modules implemented and application subdirectory, as well as their respective config files. This would enable multiple collaborators to submit new applications in parallel without having conflicts in a global db file, something that OpenFlexure does not really have to worry about.

The workflow for updating specific modules would be then to submit a pr with changes to the CAD files, as well as changing the names of the files in the config. With strict contributing rules for file names (e.g. version number must come at the end of the file, behind a _v, the latter filename updating step could potentially even be eliminated. Backwards compatibility issues could be solved by applications in each config file having the option to specify a specific version (by commit ID) required (like a python requirements.txt).

Is this what you imagined?

The use of parameters in OpenSCAD does seem extremely helpful, but converting all ipj files into an OpenSCAD design would add a lot more overhead.

I agree that this does sound like something that would be extremely useful, but I also have limited experience with webapps, so this might be something I would have to look into later.

[beniroquai]

That's exactly what I had in mind! Currently, we have a huge folder with all the STL files which need to be put in each subfolder manually. That's not a good way to organize the files, since an updated file will probably not get propagated to all the folders. Having the online configurator (similar to the one from flexuremicroscope) could pack a ZIP with all necessary printing files and print all components which are necessary to buy. Some Javascript tool which has a YAML or even a simple google spreadsheet somewhere. It compiles the ZIP by packing everything from one always-up-to-date STL files channel - done. In the OFM, the files are getting build at runtime it seems. That's awesome, but doesn't work for us of course.

Would be cool if you could help out here. I have basically no web programming skills ;-)

Am Do., 11. Juni 2020 um 21:33 Uhr schrieb Alec notifications@github.com:

Thanks! This looks like an issue with how SolidWorks imported the ipj file, it looks correct on Autodesk Inventor, and converting it to a STEP file fixed all issues.

As for your idea regarding a similar online configurator to that of OpenFlexure, how do you envision this working with the modular design philosophy of UC2? One json/yaml file is created per module with config options, as well as the rules for which file to include for each respective choice? Some sort of yaml file in each application folder specifies which modules to include for the application, which parameters might be required, and which might be flexible?

The webapp could scan through each module as well as each application to build it's database, and a dropdown/link could specify whether or not the user was looking to print a series of modules or a specific application?

This would require building

• a webapp (hosted on github pages? so using JS only)
• A config file for each module in CAD/
• A config file for each application in APPLICATIONS/

The workflow for submitting a new application would then be a pull request including just the additional modules implemented and application subdirectory, as well as their respective config files. This would enable multiple collaborators to submit new applications in parallel without having conflicts in a global db file, something that OpenFlexure does not really have to worry about.

The workflow for updating specific modules would be then to submit a pr with changes to the CAD files, as well as changing the names of the files in the config. With strict contributing rules for file names (e.g. version number must come at the end of the file, behind a _v, the latter filename updating step could potentially even be eliminated. Backwards compatibility issues could be solved by applications in each config file having the option to specify a specific version (by commit ID) required (like a python requirements.txt).

Is this what you imagined?

The use of parameters in OpenSCAD does seem extremely helpful, but converting all ipj files into an OpenSCAD design would add a lot more overhead.

I agree that this does sound like something that would be extremely useful, but I also have limited experience with webapps, so this might be something I would have to look into later.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bionanoimaging/UC2-GIT/issues/32#issuecomment-642886285, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBE5OHFZDX5EBWONTXOJJLRWEWO3ANCNFSM4N2V6MRQ .

[AlecVercruysse]

It's taken me a bit due to my limited experience with webapps, but I do have a working example now, built with vue.js. It's hosted for free on netifly, and can alternatively be run locally by cloning my repository: AlecVercruysse/uc2-configurator.

The readme and issues tab specify some simple improvements that could be made, in my opinion, that would definitely be nice to have.

In addition, a lot of configuration files still need to be generated. In the interest of time, I've only really focused on the files included in the incubator microscope (I plan on using a similar setup in an FPM project). An example config looks like this: (Baseplate config.json). I have also hacked together a python script that helped me quickly write json-compliant config files, but it probably takes a second to poke around in the (minimal) code to see how exactly to use it: CAD/config_generator.py. Right now I think I have a similar config generator in the APPLICATION folder of my fork to (poorly) auto-generate application config files. Once again, the only one I've focused on is that of the Incubator Microscope.

If this app looks like something you'd be interested in, however, there's the additional problem of generating all configuration files, and then transferring ownership of the website repository to the organization maintaining UC2.

For now, maybe it would be helpful to have a separate branch in UC2-GIT where config files can be fleshed out? The website could start tracking that branch with very minimal effort.

[beniroquai]

Wow! I'm truly impressed by your work!! I took some days off and just installed npm on my computer to start digging into the code a bit...then I got your email. Brilant! Thanks a lot! @b286 and @renerichter were already discussing how we could automize the build process of the json files you supported in your fork. I guess in Autodesk Inventor there are plenty of options to export the bill of material as an excel file somehow. From there, we can build the json files to feed your UC2-selector. I'll review that. In the meantime, we could think about a place where to put this APP.

@b286 do you think we could always refer to this tool instead of the CAD-folders or would you like to keep them and instead "just" refer to the always-up-to-date STLs using this tool. Maybe it's the best idea to start a new repo with a gigantic folder of STLs and the selector which packages the zips. The UC2-GIT links to the selector and provides the documentation on the build and assembly process.

What do you think?

[AlecVercruysse]

Thank you for the kind words!

I think keeping the stl files under each respective module subfolder in this repository would be ideal, since it facilitates easy contribution of additional modules. Another repo, in my mind, just adds unnecessary additional complexity.

I do think that this tool could be used to replace the STL subfolder of each APPLICATION. This could end the need to propagate a change in the CAD of a single module to every copy of that file lying somewhere in the applications. If an application needs an older version of a module for compatibility reasons, we could implement such an option in a config file, where the website serves the file at the commit specified in the config. This would really put the full magic of git to use, rather than having to old copies of files laying around in the repo. Do you believe this is feasible?

[beniroquai]

Here is an example billof-material Excel-file of a random Autodesk Inventor Assembly file: UC2_test_BOM_export_Inventor.xlsx Unfortunately it's in German, but this can be changed. From there, we only need to build a python file which converts the excel into the yaml with proper links to the file. There may be some additional tweaks we need to perform, but I think that's it, right? We could keep it in the main repo and remove all CAD subfolders. I'll review that :)

[beniroquai]

@AlecVercruysse I had the chance to go through your code and I must say I really like what you did! Nevertheless, I think we could even simplify the effort to organise the files. Based on the excel sheet I attached previously, we could "just" link the external/internal files with your configurator. It selects the proper file names from a dedicated always up-to-date STL folder and zips it as a folder. What do you think?

@b286 mentioned that there might be some issues with certain parts in the exported BOM such as screws, cellphones etc. But if our naming convention is correct, we could account for that in the config.json build file.

Additional errors may occur if:

• Baseplate units may differ (manually adjustment necessary?)
• ...?

We could also link descriptions exported from Autodesk Inventor. And maybe even the thumbnails in the .xclx-file.

I'm defintely not an expert for VUE, so I need to ask you if you think it's feasible to adjust the code such, that it still checks for available config.json files in all subfolders and then links it to the accumulated STLs in the root-folder? Should be fine it seems.

[beniroquai]

So @b286 and me agreed on the following:

We have this google-sheet which has all the information about the setups, assemblies, modules and parts. I'll try to write a python script which generates a config.json for each assembly and module and automatically copies it in each subfolder. I think it's then best if we have one big folder with all STLs which links to the json files.

As far as I understand, there does not need to be a large change in your programm to follow this path, right? Additionally, I would really love to integrate some pictures into the configurator. Do you think that's feasible @AlecVercruysse ?

Thanks a lot! :)

[kasbah]

@AlecVercruysse I'd be grateful if you could give the ofm-stl-selector a go, reconfiguring it to use the UC2 STLs. What you have to do:

1. git clone https://gitlab.com/openflexure/ofm-stl-selector && cd ofm-stl-selector
2. Run npm install
3. Remove everything in stl_options.json and replace it with UC2 data, I documented the format in a comment recently
4. Run ./pre_compute
5. Run npm run serve to see the VueJS dev server on http://localhost:8080

There would be some additional steps to remove some of the Openflexure branding, but it shouldn't be too hard and I would like to let this web-app be a generic thing. Looks like we picked a lot of the same tech to solve this problem so it would be good to work together.

If you run into any issues with any of this, let me know.

[AlecVercruysse]

@kasbah, thanks for reaching out! I loved the design of your webapp and your use of Vue inspired me to go explore it myself!

I went ahead and wrote a sample config (stl_options.json.txt, added .txt because I couldn't upload a .json file to this comment) for some basic presets, and changed the build-url in StlSelector.vue to my fork of UC2-GIT. The precompute works and I get my files from the download page. I also really like what you did with the router, and was hoping to include that at one point in uc2-configurator.

The fundamental problem in applying this webapp to UC2-GIT, in my opinion, is that it was not designed for a modular system, where each module has it's own configuration, and some modules are not included. A workaround would be to have hierarchical configuration options, and include a "root option" to include separate modules. Like you touched on in a comment in issue #2 on ofm-stl-selector, I believe that implementing truly hierarchical modules would require an SMT solver. Although I've never worked with one before, I believe it would greatly increase complexity. In uc2-configurator, the "need" for hierarchical options can be bypassed by implementing options on a per-module basis.

Furthermore, even with hierarchical options, there would ultimately be no way to add two of a single module. If, for example, a 4x2 and 4x4 baseplate were required, or multiple mirror modules with different inserts, there would be no way to account for this in a design not centered around modularity, where modules can be added and removed as pleased.

Lastly, serving files from github allows for specifying a required version of an STL by commit id in an application config. While uc2-configurator does not yet support that, I designed it with such a potential goal in mind, and believe that it would be rather simple to implement. I'm not sure if ofm-stl-selector would ever have this functionality, as it seems you do not serve your files from a git repo.

[kasbah]

Thanks for trying it out and a thorough descriptions of the issues you see.

I'd be very interested to add the ability to have multiples of the same STL in the output. Adding hierarchies, I have come to believe, will actually reduce the computational complexity. Right now the system thinks that every option has an impact on every other option and working it is close to O(n!) (factorial complexity), but with a hierarchy the number of options (n) is reduced to the first level of the hierarchy: the set of options it needs to care about for a given computation should be drastically reduced. We may be able to avoid an SMT solver yet.

You should be able to serve the files from any http(s) address, including Github. The only problem is I don't think Github allows cross-origin requests on its resources so you have to use a proxy or a proxy service like https://raw.githack.com. But you will have to do that for your own implementation too.

[AlecVercruysse]

@kasbah I believe the issues with multiple modules is not necessarily needing to add duplicates of a file (in fact, I specifically choose to remove duplicates in uc2-configurator), but rather being able to specify separate configurations for multiple versions of a module.

Additionally, while I'm not a CS major so I could definitely be wrong, I believe that uc2-configurator's file-inclusion calculation is O(n) for n being possibly included files. I don't think the computation time is much of an issue here though. By complexity, I was mainly referring to the complexity of the code and the project itself. Adding hierarchy and complex rules for the inclusion of sub-options and files seems like it would require a fundamental change to how configuration files are specified as well. All this, in my opinion, further introduces the possibility of hard to catch bugs in either the parsing config files, or even in future contributors' config files themselves (in the case of UC2-GIT). If adding hierarchical options is something that you believe is feasible and beneficial within ofm-stl-selector, however, please let me know as I would be happy to help contribute to the project.

Ultimately, I do think that a user interface centered around adding and deleting modules, along with simple modular configs, is more adapted to the specific needs of UC2. If you believe that this could be somehow adapted into ofm-stl-selector, please let me know.

I also did not run into any CORS issues using github. A network analysis of uc2-configurator reveals the Access-Control-Allow-Origin: * header present in github's response.

I will also make sure to credit and link ofm-stl-selector in uc2-configurator, so that potential users and contributors are directed to your tool!

[AlecVercruysse]

@beniroquai I see how including all .stl files in a root folder is beneficial, especially so that multiple copies of common stls like the lid or base are not present in the repo. I do believe that it is still useful to include some version of the CAD/ folder that includes readmes, images, and the config files for each module (although I assume CAD would have to be renamed).

I've updated uc2-configurator to reflect this (I did not rename CAD/ nor remove any CAD/*/STL subdirs, but all stl files now reside in STL/ in AlecVercruysse/UC2-fork).

As for images, what sort of images were you thinking, and where? The UC2 logo, images of selected applications, and/or images of each module? That is certainly feasible, but I would be interested to hear what you had in mind.

[beniroquai]

Dear @AlecVercruysse,

thanks again for your great effort to make this work. I started working on a simple python script which converts the xlsxfile from gdrive into a class-based Application-Module-Part converter. Before I start converting this structure to the JSON file you requested, I wanted to kindly ask you if this suits your needs ;-) You can find the files here.

Next step would be the creation of JSON files. I was asking myself if we should populate them to the CAD subfolders or also keep them together with all STL files somewhere else?

[AlecVercruysse]

@beniroquai,

The python script looks good! I see that you include both 3d printed and non-3d printed parts in the script. I can look into adding configuration options, and some sort of BOM export for non-printed parts, later this week. If you'd like, I could also look into the actual json export later this week as well. Most of the commented code at the end of generate_config_from_database.py is not required for the json export. The first block attempts to find filenames in the APPLICATIONS/*/STL/ subdir that are unique to a single module in order to include that module in the application config. This is no longer needed since your code includes every module more correctly based off the spreadsheet. The second block uses a "dumb" regex to look for some sort of description in the readme for the application, which ends up working around half the time (better descriptions can be written manually). The only code that writes the file is:

         with open(d.name + "/config.json", "w") as j:
             data = {
                 "type":"application",
                 "description" : desc,
                 "modules": get_modules(d.name)
                 }
             json.dump(data, j, indent=2)

In my opinion, it would make the most sense to put all application json files in the subdir for each application. Following that convention, the module json files would go in the CAD subfolders along with the readme for the module and any images included in the readme/config file. I think there are a number of benefits to this scheme, including the fact that every config file could remain named as config.json, and that when editing a module, a contributor largely has to work in a single subfolder to accomplish their work (other than potentially editing files from the master stl folder).

[beniroquai]

Sounds very good to me! :) I would love to do it myself, but my schedule is a little bit tight this week. I hope to work on it next week again :)

Am Mi., 8. Juli 2020 um 19:25 Uhr schrieb Alec notifications@github.com:

@beniroquai https://github.com/beniroquai,

The python script looks good! I see that you include both 3d printed and non-3d printed parts in the script. I can look into adding configuration options, and some sort of BOM export for non-printed parts, later this week. If you'd like, I could also look into the actual json export later this week as well. Most of the commented code at the end of generate_config_from_database.py is not required for the json export. The first block attempts to find filenames in the APPLICATIONS/*/STL/ subdir that are unique to a single module in order to include that module in the application config. This is no longer needed since your code includes every module more correctly based off the spreadsheet. The second block uses a "dumb" regex to look for some sort of description in the readme for the application, which ends up working around half the time (better descriptions can be written manually). The only code that writes the file is:

     with open(d.name + "/config.json", "w") as j:
         data = {
             "type":"application",
             "description" : desc,
             "modules": get_modules(d.name)
             }
         json.dump(data, j, indent=2)

In my opinion, it would make the most sense to put all application json files in the subdir for each application. Following that convention, the module json files would go in the CAD subfolders along with the readme for the module and any images included in the readme/config file. I think there are a number of benefits to this scheme, including the fact that every config file could remain named as config.json, and that when editing a module, a contributor largely has to work in a single subfolder to accomplish their work (other than potentially editing files from the master stl folder).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bionanoimaging/UC2-GIT/issues/32#issuecomment-655652800, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBE5OHM6ZNI2NVZ4UUWKELR2STXXANCNFSM4N2V6MRQ .

[beniroquai]

@AlecVercruysse we have added all .stl files and .ipt files to the repo. So I think we could continue to work on the online tool :-)

You can find all files here.

Since you can export all .stl-files at once, all of them have a prefix: Assembly_ALL_PARTS_FOR_EXPORT_*.

[AlecVercruysse]

My edits to the python script can be found on my fork. It writes module configs, but not yet the application configs. All config files on my fork are updated with this script.

Please note that for the module configs, the spreadsheet does not define what options need to be created (and therefore, what files to include). This is ultimately something that needs to be created manually, or "semi-manually" with some sort of helper script, I believe. It does add the price, extra parts, the github link, and the image link to the config. I can use these attributes to eventually implement new functionality in the webapp.

Also, the spreadsheet seems to not include many modules. such as the XYZ-Stage for example.

Lastly, please see my comment on https://github.com/AlecVercruysse/UC2-GIT/commit/2fedf5b649361c77136a37925e64e86619c50712, which reveals discrepancies between the STLs included for each module in their respective subdirs, and what is given in the CAD/RAW/STL dir. For now, to avoid 404 errors when using the webapp, I've continued to use the STL/ dir in my fork, instead of the Assembly_ALL_PARTS_FOR_EXPORT_ files.

[beniroquai]

Cool! I'll definitely give it a try in the next days. I was also trying to get the nested class structure of the applications into a proper JSON file, but I wasn't lucky yet. Need to have a look. I've updated the python script, but it producs weird stuff. I'm aiming for exporting the proper json files in each application folder, right?

Regarding options in the modules, do you wish to put this into the google sheet to? More like an updatable database then.. @b286 would you mind thinking about it?

Regarding individual STLs; Yes, certainly there are some parts missing. All baseplate (e.g. 3x1, 4x2) combinations for example @b286 would you mind to add this too :)

[AlecVercruysse]

I can work on writing the application configs with the script next, if you'd like. Yes, every config file goes in the root directory of the specific application (e.g. APPLICATIONS/APP_Incubator_Microscope/config.json).

It might be a good idea to add the file locations (and options) into that spreadsheet. Ultimately, I think it would be ideal if every ASSEMBLY_* folder in CAD/ corresponded to a single entry in the spreadsheet. Multiple instances of ASSEMBLY_CUBE_Lens (e.g. +40, +100, -50) would then be removed. Special configurations (e.g. the inclusion of STL files dependent on some options) would be specified for each application if needed. We could also have dynamically determined (options-dependent) "extra parts", analogous to our dynamicFiles and fixedFiles now, to take care of issues such as the focal length of the lens that needs to be included in the BOM. I am relatively unfamiliar with the structure of the spreadsheet especially since I did not write the code that extracts the data from it, but I would imaging that these changes would suffice:

• fixedFiles would specify the paths (relative to the root dir of the repo) of the fixed STL files to include.
• dynamicFiles would specify the the option-dependent STL files to include. An example format for each entry could read: <path>:<condition1>=[<option1>,option2>],<condition2>=[<option1>,<option2>],...
• fixedExtraParts would contain the always-included non-printable entries that the current part column contains
  • the Part column and Printable column could be deleted
  • dyanmicExtraParts would contain the option-dependent extra parts to include, with an identical format to that of dynamicFiles, replacing <path> with <part name>.

Please let me know if you have another idea. Although I would be happy to continue work on the configurator, I cannot make all these changes to the spreadsheet, due to both time constraints and because I am not familiar with how you might want to structure the options for all your modules.