V1.0 prep

[wincowgerDEV]

@zsteinmetz, creating this PR so that we can have everything in one place. Below I am bringing over comments and check boxes from the other 3 PRs and assigning each of us to tasks. Would it be alright if I merge all these to V1.0 prep?

Package Down Documentation (@wincowgerDEV) https://github.com/wincowgerDEV/OpenSpecy-package/pull/126

• [ ] Update SOP with new functionality and explanations.
• [ ] Update homepage.
• [x] Customize the website to look its best.
• [ ] Set URL address to something like OpenSpecy.org.

Functions for managing spectra for v1.0 (@zsteinmetz) https://github.com/wincowgerDEV/OpenSpecy-package/pull/125

• [x] finalize functions to align and concatenate spectra required by #124
• [x] add test routines
• [x] update docs
• [x] check if everything works as expected

Reading functions for v1.0 (@zsteinmetz) https://github.com/wincowgerDEV/OpenSpecy-package/pull/124

• [x] functions to align and concatenate spectra required before multiple spectra could be read at once, e.g. from a zip file
• [x] finalize reading functions
• [x] add test routines and update sample files
• [x] update docs
• [x] check if everything works as expected

Functions to be renamed/changed (@zsteinmetz)

• [x] sample.OpenSpecy() -> sample_spec()
• [x] correlate_spectra() -> cor_spec()
• [x] collapse_spectra() -> collapse_spec()
• [x] process_spectra() -> process_spec()
• [x] identify_spectra() -> ident_spec()
• [x] characterize_particles() -> def_features()
• [x] heatmap_OpenSpecy() -> heatmap_spec()
• [x] plot_OpenSpecy() -> plotly_spec()
• [x] to_hyperSpec() -> as_hyperSpec()
• [x] subtr_bg() -> subtr_baseline()
• [x] replace magrittr pipe %>% with native R pipe |> were possible
• [x] replace object argument with x where possible
• [x] update run_app()

General checks and CRAN compliance (@zsteinmetz)

• [x] passing urlchecker::url_check()
• [x] passing GitHub checks
• [x] passing devtools::check(remote = TRUE, manual = TRUE)
• [x] passing devtools::check_win_devel()
• [x] optimizing/adding tests (still missing for read_envi and read_envi_nic)
• [x] check docs for consistency and spelling mistakes
• [x] finalize dependencies in DESCRIPTION
• [x] update NEWS.md
• [x] update cran-comments.md

[wincowgerDEV]

Thanks for doing that! Good points about the app testing. In the long run, I think we will want to create some modules or try out other ways for testing the app but currently it's all pretty manual. I'll work on the SOP, if I can't get to it today, I should be able to tomorrow.

[wincowgerDEV]

@zsteinmetz, writing the SOP I realized that the identification pipeline is a little clunky. I really like that the user just has one function for the process_spec and wondering if we should do a single monolith for the identification, too, that basically nests cor_spec and ident_spec into a logical arrangement and the user just provides the parameters that they need for each in one function. This will also allow us to integrate additional options like AI based identification later on and the user just needs to remember that they have to use that one function for all of their identification needs. What do you think?

[zsteinmetz]

@wincowgerDEV Yeah, great idea! This caught my eye as well when I reviewed the code but I apparently forgot about it upon working on other stuff. Since we already have an ident_spec() function, the monolithic function could become our new match_spec(). What do you think?

I won't have much time until mid/end of next week. Do you like to fire up a first draft and I have a second look afterwards?

[codecov-commenter]

Codecov Report

Patch has no changes to coverable lines.

:exclamation: Current head 77a4689 differs from pull request most recent head f8afebd. Consider uploading reports for the commit f8afebd to get more accurate results

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

:loudspeaker: Thoughts on this report? Let us know!.

[wincowgerDEV]

@wincowgerDEV Yeah, great idea! This caught my eye as well when I reviewed the code but I apparently forgot about it upon working on other stuff. Since we already have an ident_spec() function, the monolithic function could become our new match_spec(). What do you think?

I won't have much time until mid/end of next week. Do you like to fire up a first draft and I have a second look afterwards?

Yeah I'll give it a go today!

[wincowgerDEV]

@zsteinmetz just want to pass this by you real quick. With the process_spec function, I was thinking that we try to set the defaults to whatever we think will maximize the signal to noise and streamline the identification process. I want the user to be able to just run process_spec(spectrum) then the match_spec and get a good result most of the time quickly. I changed the defaults to conduct the absolute derivative with make_rel at the end because in most cases I have tested that algorithm does everything you need super quickly. Let me know what you think, I can change it back.

[zsteinmetz]

@zsteinmetz just want to pass this by you real quick. With the process_spec function, I was thinking that we try to set the defaults to whatever we think will maximize the signal to noise and streamline the identification process. I want the user to be able to just run process_spec(spectrum) then the match_spec and get a good result most of the time quickly. I changed the defaults to conduct the absolute derivative with make_rel at the end because in most cases I have tested that algorithm does everything you need super quickly. Let me know what you think, I can change it back.

Sounds good to me. No objections!

[wincowgerDEV]

Appreciate ya @zsteinmetz!

[zsteinmetz]

@wincowgerDEV What do you think when we might be ready to merge into main? I am asking because I actually don't like to see the main branch fail. Currently it's only because of that GitHub Action deprecation but still. If we need more time here, no problem. Then I just cherry pick the GitHub Action commits and merge them already before merging the whole thing. If it's only documentation left, I guess we could merge this one here already and continue working on the SOP on this or another branch.

[zsteinmetz]

@wincowgerDEV I just split up the SOP vignette into three separate ones: the main SOP, the advanced spec part, and the SpectraGryph stuff. The idea behind was (1) to improve clarity also with respect to packagedown and (2) to be more flexible which vignettes to exclude for the CRAN package to keep the file size down. For now, I only added the main SOP to be submitted to CRAN excluding the two other ones that only go to packagedown. What do you think? I can revert it if it doesn't make sense to you, of course.

[wincowgerDEV]

@wincowgerDEV What do you think when we might be ready to merge into main? I am asking because I actually don't like to see the main branch fail. Currently it's only because of that GitHub Action deprecation but still. If we need more time here, no problem. Then I just cherry pick the GitHub Action commits and merge them already before merging the whole thing. If it's only documentation left, I guess we could merge this one here already and continue working on the SOP on this or another branch.

I think we merge with main asap 🙂. Just vignettes left. I'm gonna finish the vignettes today. Love what you did with the split too, that's super handy. There will likely still be some small bugs to shake out but the community can help us find them more easily once it is in main. You wanna do the honors of merging?

[zsteinmetz]

Sure! I'm going to merge as soon as all checks have passed.

We could even think about splitting the package and app parts in the current SOP vignette. This would make the actual package vignette super small, because we don't need all the screenshots, and would give us more space for sample files and so on. The app vignette could then go to packagedown only.

[wincowgerDEV]

That's not a bad idea. Was going back and forth about whether to do that. I think we originally kept them together so that the app could expedite the learning process for the package but I totally get that now we have a lot more app features and those each have images and that can make a big file size for the package. Maybe in the app vignette and the package vignette we try to maintain the same headings to make it easy to search between the two?

On Wed, Aug 30, 2023, 7:52 AM Zacharias Steinmetz @.***> wrote:

Sure! I'm going to merge as soon as all checks have passed.

We could even think about splitting the package and app parts in the current SOP vignette. This would make the actual package vignette super small, because we don't need all the screenshots, and would give us more space for sample files and so on. The app vignette could then go to packagedown only.

— Reply to this email directly, view it on GitHub https://github.com/wincowgerDEV/OpenSpecy-package/pull/129#issuecomment-1699335129, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGMUJU22YOK2OQ36FAE5K4DXX5HRPANCNFSM6AAAAAA2C62GJI . You are receiving this because you were mentioned.Message ID: @.***>

[zsteinmetz]

Maybe in the app vignette and the package vignette we try to maintain the same headings to make it easy to search between the two?

Sounds good! We could also reference the package functions in the app vignette and vice versa.

[wincowgerDEV]

Yeah that's true, maybe even add a link to the header on the other package vignette too! I like keeping the package as small as possible, there may come a time we want to make another big push in a few years and we will want the space for test files or porting over other people's 1000 line codes 😂.

On Wed, Aug 30, 2023, 8:00 AM Zacharias Steinmetz @.***> wrote:

Maybe in the app vignette and the package vignette we try to maintain the same headings to make it easy to search between the two?

Sounds good! We could also reference the package functions in the app vignette and vice versa.

— Reply to this email directly, view it on GitHub https://github.com/wincowgerDEV/OpenSpecy-package/pull/129#issuecomment-1699349628, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGMUJU44YXP2Z5UONO7ZEVDXX5IRNANCNFSM6AAAAAA2C62GJI . You are receiving this because you were mentioned.Message ID: @.***>