Add support for Landsat Collection 2

[XavierCLL]

Hi,

Is there a plan to support Landsat Collection 2? does it require a lot of changes in the equations or in its settings?

with minimum changes, I could run python-fmask in an image of Landsat 8 C2, but the result was bad:

Thanks, regards, Xavier

[neilflood]

Hi Xavier,

Nice to hear from you :-)

This is a good point. I had not yet tested anything from Collection-2, but good to know that some changes are required. I have only recently started even looking at Collection-2, for some other reasons, but I will now start to think about how it affects python-fmask, too.

I can't promise to do anything immediately, but I will certainly be looking at it. I will definitely plan to add Collection-2 support, unless there is some really serious barrier.

Let me know if this becomes urgent for you.

cheers Neil

[XavierCLL]

Hi Neil!

Thanks for your answer, Sooner is better, Collection 2 is a very good product and they eventually will drop Collection 1. (and maybe merge with the last version of the algorithms https://github.com/ubarsc/python-fmask4 ?)

Thanks so much Neil and let me know any help I can do for that.

Xavier

[petebunting]

Hi all,

As far as I am aware the collection 1 cloud masking was from FMask (https://www.usgs.gov/core-science-systems/nli/landsat/cfmask-algorithm) and that has not changed in collection-2.

Therefore you could just use the QA band provided rather than running python-fmask.

Cheers Pete

Sent from my iPhone

On 27 May 2021, at 19:44, Xavier C. Llano @.***> wrote:

 Hi Neil!

Thanks for your answer, Sooner is better, Collection 2 is a very good product and they eventually will drop Collection 1. (and maybe merge with the last version of the algorithms https://github.com/ubarsc/python-fmask4 ?)

Thanks so much Neil and let me know any help I can do for that.

Xavier

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[neilflood]

Hi Pete,

this is a very good point. A few years ago I added a note about this to the front page at http://www.pythonfmask.org/en/latest/#usgs-landsat, suggesting that people might prefer just to use that. However, there has been continued interest in this package, so I have just kept going.

I also did some small tests on this, ages ago, and it seemed like their cloud mask was not as good as the output of this package, even though they should be roughly the same. It was partly confusing because they supply it as 3 cloud confidence levels, so not exactly sure how to threshold this, but nothing seemed quite right. I was only testing on Australian data, so I don't know how general my results are, but I was worried enough that our group kept using this package.

I don't really know what to do about this. I would be interested to hear any thoughts from @XavierCLL ?

On the subject of Fmask4 - there is also a note on this at http://www.pythonfmask.org/en/latest/#later-fmask4-methods. I also don't really know what to do about that.

Sorry, I am not much use, am I. :-) However, I will definitely continue on adapting this code for Collection-2.

Neil

[XavierCLL]

Hi Neil,

In my case, Colombia works very well for collection 1, we (me and https://github.com/SMByC) have been using this library through the command line by python scripts or Qgis plugin that I developed. The use of the Qgis plugin is when we have few scenes, a short period to analyze, so every pixel matters, then we masking each scene adjusting the parameter in the equations to improve the cloud/shadow detection with other masks like pixel QA. That is possible because Qgis plugins work with python and this lib is the only implementation of fmask in python. So, for us, it is so important that this lib continues with development.

I didn't know that the fmask4 doesn't improve the results, in your case, but good to know.

I really appreciate your time and work in adapting this to collection 2, let me know if you need any help.

Thanks, Xavier

[neilflood]

I just tested the existing code on a Landsat-8 image from Collection-2. I found that I did not need to modify anything, and it all worked fine.

Could you let me know a bit more detail of what you found? You mention that you needed to modify a few things - what changes did you need? And is the example you showed above definitely different to what it does with the same image from Collection-1?

many thanks Neil

[neilflood]

@XavierCLL ,

I just realised something. Are you talking about trying to use the Level 2 product? If so, that probably won't work without some serious modifications. The whole algorithm was created for Level 1, i.e. top-of-atmosphere reflectance, but the Level 2 product is surface reflectance. As such, the values are generally somewhat lower, by different amounts in different bands, so some of the thresholds will need to be adjusted, and some of the tests (like the "whiteness" test) will be a bit different.

My test above was using Level1, from Collection 2, and it all works fine.

Is this the problem, or am I just guessing? :-)

At the moment, I have no plans to adapt to use Level 2, but Level 1 will continue to be available in Collection 2.

Neil

[XavierCLL]

Hi @neilflood ,

Yes, that was the confusion, sorry to not be clear at the beginning, and yes I was talking about Collection 2 Level 2. I didn't answer your test above because I thought that was level 2 and I thought that I did something wrong to run it. But now it is clear, the level 2 is very different and requires a lot of changes for run python-fmask.

Thanks for the test and confirming the issues with level 2

Regards, Xavier

[neilflood]

Hi Xavier,

OK, that makes sense now. No problem at all, I am just glad to have your enthusiasm.

Neil