[PRE REVIEW]: A Short Introduction to PF: A C++ Library for Particle Filtering

[whedon]

Submitting author: @tbrown122387 (Taylor Brown) Repository: https://github.com/tbrown122387/pf Version: v1.0.1 Editor: @diehlpk Reviewers: @ziotom78, @andremrsantos Managing EiC: Daniel S. Katz

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @tbrown122387. Currently, there isn't an JOSS editor assigned to your paper.

@tbrown122387 if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Reference check summary:

OK DOIs

- 10.1080/10618600.2016.1172487 is OK
- 10.1007/978-1-4684-9393-1 is OK
- 10.1111/j.1467-9868.2009.00736.x is OK
- 10.1063/1.1699114 is OK
- 10.1111/1467-9868.00363 is OK
- 10.1214/aoms/1177699147 is OK
- 10.1109/5.18626 is OK
- 10.1049/ip-f-2.1993.0015 is OK
- 10.1109/TSP.2005.849185 is OK
- 10.3150/14-BEJ666 is OK
- 10.2139/ssrn.2386371 is OK

MISSING DOIs

- https://doi.org/10.1093/biomet/57.1.97 may be missing for title: Monte Carlo Sampling Methods Using Markov Chains and Their Applications
- https://doi.org/10.1109/9780470544334.ch9 may be missing for title: A New Approach to Linear Filtering and Prediction Problems
- https://doi.org/10.1080/01621459.1999.10474153 may be missing for title: Filtering via Simulation: Auxiliary Particle Filters

INVALID DOIs

- None

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[danielskatz]

@whedon check repository

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.84  T=0.86 s (592.6 files/s, 93886.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
HTML                           231           1519           1285          34026
C/C++ Header                    18           4526           3047          16726
TeX                             69           2241             65           8055
JavaScript                     130            134            114           3353
CSS                              4            331             80           1603
C++                              9            293            201           1123
CMake                           42            129             84            612
C                                1            114             50            507
make                             2             93             56            157
Markdown                         2             40              0             73
YAML                             1              1              1              1
-------------------------------------------------------------------------------
SUM:                           509           9421           4983          66236
-------------------------------------------------------------------------------

Statistical information for the repository '2559' was gathered on 2020/08/09.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Taylor                          96         45188          16286          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Taylor                    30222           66.9          5.6               11.36

[diehlpk]

@danielskatz I would like to edit this paper.

[diehlpk]

@whedon assign @diehlpk as editor

[whedon]

OK, the editor is @diehlpk

[diehlpk]

Reference check summary:

OK DOIs

- 10.1080/10618600.2016.1172487 is OK
- 10.1007/978-1-4684-9393-1 is OK
- 10.1111/j.1467-9868.2009.00736.x is OK
- 10.1063/1.1699114 is OK
- 10.1111/1467-9868.00363 is OK
- 10.1214/aoms/1177699147 is OK
- 10.1109/5.18626 is OK
- 10.1049/ip-f-2.1993.0015 is OK
- 10.1109/TSP.2005.849185 is OK
- 10.3150/14-BEJ666 is OK
- 10.2139/ssrn.2386371 is OK

MISSING DOIs

- https://doi.org/10.1093/biomet/57.1.97 may be missing for title: Monte Carlo Sampling Methods Using Markov Chains and Their Applications
- https://doi.org/10.1109/9780470544334.ch9 may be missing for title: A New Approach to Linear Filtering and Prediction Problems
- https://doi.org/10.1080/01621459.1999.10474153 may be missing for title: Filtering via Simulation: Auxiliary Particle Filters

INVALID DOIs

- None

@tbrown122387 Can you please add the missing DOIs to the paper?

In addition, can you please provide your full affiliation including city, state, and country.

[diehlpk]

@hkaiser @NK-Nikunj @brycelelbach @sithhell @biddisco @gentryx would anyone be interested to review this paper?

[tbrown122387]

@diehlpk done :)

[diehlpk]

@whedon generate pdf

[whedon]

:point_right: Check article proof :page_facing_up: :point_left:

[diehlpk]

@tbrown122387 Could you please recommend some reviewers? You can name them here using their GitHub handle without the @ at the beginning.

[diehlpk]

Hi @wrathematics @betatim @jochym @dvalters @jwuttke @leios would you be interested in reviewing this paper for JOSS?

[jwuttke]

no capacity right now, sorry.

[teuben]

Not reviewing, but I'm offering one comment. I have no clue from the README.md first paragraph what this is about. Is a particle an object at some position with lots of attributes and you filter one that? If so ,say that.

[diehlpk]

Not reviewing, but I'm offering one comment. I have no clue from the README.md first paragraph what this is about. Is a particle an object at some position with lots of attributes and you filter one that? If so ,say that.

@tbrown122387 Could you please update the README.md accordingly?

[tbrown122387]

@diehlpk To include the definition of a particle? Or the definition of “particle filter?” Regarding the first one, @teuben has it right—they are just samples targeting a distribution, and the distribution could indeed describe unobserved “positions.”

[diehlpk]

@tbrown122387 I think it would be good to add the definition of particle, since different scientific fields have different meanings for particles. So I think it would be beneficial to make clear what you define as particles. I think also adding the term particle filter would help, because here different scientific fields have different interpretations.

[teuben]

If it's that general a filter, why isn't this just a table? The "particle" is then a row in the table, it has many attributes, and you can filter them. Is that description as accurate? I guess I'm asking, what make your row a particle, and not a row in a table. Are there other properties linking the rows?

[tbrown122387]

If I understand you correctly @teuben it isn't really a table. That analogy wouldn't work if each sample had dimension greater than 1, but more importantly, you're not really adding "rows."

The primary bits of data in each class are two "rows" (std::arrays). One stores samples that target the filtering distribution (I also suspect we're not using the same definition of that word, too), and the other stores weights for those samples. All of the samples and the weights change at each time point too, after you call filter, and update them based on a new piece of information.

These particles (or weighted-samples) are used to approximate the filtering distribution. Time t's filtering distribution is the distribution of the log-volatility at time t, given all of the observed information up until that time point: p(x_t | y_1, ... y_t) t moves forward, and you get to approximate the sequence of these distributions in real-time. The memory requirements don't grow, and neither does the dimension of the distribution you're targeting. How and when they fail is more of a statistical question, that's definitely outside the scope of this paper.

There are particle "smoothing" algorithms--those target the sequence p(x_1, ... x_t | y_1, ... y_t). Those have expanding dimension and memory requirements, though, and are not addressed by this software. If each x_t was scalar-valued, then you could think of time as the row index, and sample index as the column index. That would be a table I guess, but you'd also be ignoring the weights. The weights would always be one-dimensional, though, because they put weights on entire paths..

In the particular example in the paper, the samples would represent a given time t's log-volatility (x_t). That's model-specific, though. Its meaning depends on the model. Without using any financial jargon, this is important because you would get up-to-date estimates of how "risky" the stock market is.

Does this help?

[diehlpk]

Hi @jordigh @hausen @pragyansmita @conradsnicta @dvalters @abhishekbajpayee @jmbr @Volkerschmid would you be interested in reviewing this paper for JOSS?

[teuben]

@tbrown122387 I think my background is playing havoc with me, since I do (astrophysical) N-body simulations. Reading on one particular wikipedia page particle was defined as a candidate solution, and I could see that if you have a swarm of solutions and want to find the optimal one, filtering is one way to weed out the bad cases. So if it's the nomenclature, I can begin to understand your lingo. If so, would that help to add this definition near the top of what people read when encountering this code?

[jmbr]

@diehlpk I can't this time around but I remain interested in reviewing for JOSS in the future.

[tbrown122387]

@teuben yes it would. I myself have a hard time understanding any of the search results obtained from my searching "N-body simulations." I just linked in the Wiki page in the first sentence.

[diehlpk]

@tbrown122387 Could you please recommend some reviewers? You can name them here using their GitHub handle without the @ at the beginning.

@tbrown122387 Do you have some reviewers in mind?

[diehlpk]

Hi @pboesu @ziotom78 @zhampel @nespinoza @williamjameshandley @adavidzh @andremrsantos @fpl would you be interested in reviewing this paper for JOSS?

[tbrown122387]

@diehlpk I don’t have anyone in mind but I’ll do some digging once I get back from vacation.

[pboesu]

@diehlpk Unfortunately I do not have time to review this at the moment.

[adavidzh]

Hi @pboesu @ziotom78 @zhampel @nespinoza @williamjameshandley @adavidzh @andremrsantos @fpl would you be interested in reviewing this paper for JOSS?

I'm afraid I have to sit this one out, @diehlpk. Thank you for asking.

[nespinoza]

Thanks for the invite to review this --- I will have to decline @diehlpk, however.

[tbrown122387]

I searched "statist*" here and I noticed several editors who have experience with statistical techniques: fboehm, mikldk, mjsottile, marcosvital, usethedata @diehlpk you are the only editor who mentions c++ explicitly in his/her bio on that page, although many editors have github repos whose primary language is c++

[diehlpk]

@tbrown122387 I asked for people of your community to review this paper? The editors do normally do not have the capacity to review papers and edit them.

[tbrown122387]

@diehlpk understood. There are no members of my professional network that are familiar with JOSS’ requirements, so I have no suggestions.

[conradsnicta]

@diehlpk Apologies, I currently don't have the bandwidth for it; i have a newborn and a full time job, sucking up all my time. The paper mentions "financial time series", so perhaps @eddelbuettel might be interested.

[diehlpk]

@diehlpk Apologies, I currently don't have the bandwidth for it; i have a newborn and a full time job, sucking up all my time. The paper mentions "financial time series", so perhaps @eddelbuettel might be interested.

Thanks for the update. Good luck with the newborn.

[ziotom78]

Hi @pboesu @ziotom78 @zhampel @nespinoza @williamjameshandley @adavidzh @andremrsantos @fpl would you be interested in reviewing this paper for JOSS?

Hi @diehlpk , sorry for the delay, but I'm currently on vacation till mid-September. I could work on this only once I came back to work, don't know if this fits the time scale of this review.

[diehlpk]

@ziotom78 Thanks for your positive response. It will be fine if you start after your vacation with the review.

[diehlpk]

@whedon assign @ziotom78 as reviewer

[whedon]

OK, @ziotom78 is now a reviewer

[andremrsantos]

Hi @pboesu @ziotom78 @zhampel @nespinoza @williamjameshandley @adavidzh @andremrsantos @fpl would you be interested in reviewing this paper for JOSS?

Sorry for the delay. I would be glad to review the paper if you're still accepting reviewers. If so, could you please link me to the journal review criteria and guidelines?

[diehlpk]

@andremrsantos

Please find the review criteria here

https://joss.readthedocs.io/en/latest/review_criteria.html

and the review checklist here

https://joss.readthedocs.io/en/latest/review_checklist.html

[diehlpk]

@whedon add @andremrsantos as reviewer

[whedon]

OK, @andremrsantos is now a reviewer

[diehlpk]

@whedon start review

[whedon]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/2599.