macports / macports-webapp

Web application to display information about ports, build history and installation statistics
https://ports.macports.org/
BSD 2-Clause "Simplified" License
53 stars 25 forks source link

Valid ports are shown as deleted #351

Closed Dave-Allured closed 1 year ago

Dave-Allured commented 1 year ago

2022 November 19:

Webapp on the Ports website shows some valid ports as "deleted". For example, go to the URL for hdf5:

https://ports.macports.org/port/hdf5/details/

This port has been deleted. This page will exist till another port with the name "hdf5" comes into existence.

Related, the search box does not find this port. It finds ports with hdf5 in the name, but not hdf5 itself.

jmroot commented 1 year ago

I gather this is because the MacPorts installation backing the web app is outdated, and so fails to parse portfiles that use features introduced in 2.8.

arjunsalyan commented 1 year ago

The site is now running with MacPorts 2.8.0 on the machine. I am unable to find hdf5 in parsed or failed ports in the PortIndex used to populate the website. Can it be due to faking of the platform to macosx_19_i386?

jmroot commented 1 year ago

It's not mentioned at all in the portindex output? If there was a problem due to faking the platform it would generally manifest as a parse failure.

arjunsalyan commented 1 year ago

Nope, it is not in the PortIndex output. By the fake platform I was thinking maybe it is not supposed to be available on that particular version? There are a few ports that do not appear in the faked PortIndex.

arjunsalyan commented 1 year ago

@Dave-Allured do you know any other examples of ports which show incorrect information?

jmroot commented 1 year ago

All ports should exist on all platforms, they may just not be installable on some (or may fail to parse, which is a bug that should be fixed.) The only way portindex should produce neither a PortIndex entry or an error is if the Portfile is missing from the tree.

jmroot commented 1 year ago

qt5-qtbase is also showing as deleted, for one.

jmroot commented 1 year ago

There must be quite a few more, as my local port count is 30,089 while the count shown on the web app is 29,580.

arjunsalyan commented 1 year ago

qt5 ports show up in parsing errors

Failed to parse file aqua/qt5/Portfile with subport 'qt5-qtbase': no usable SDK can be found
jmroot commented 1 year ago

OK, the Qt* parse failures should be fixed now.

arjunsalyan commented 1 year ago

Okay there is hdf5, for some reason my search over PortIndex.quick wasn't showing any results. Since, it is there in the portindex now, these inconsistencies will get fixed during the next full database sync.

But the cause for it seems to be what Joshua suggested about the old MacPorts version. Until last week the machine had MP 2.6.8. At some point there would have been a parse failure, and the port got marked as deleted. Incremental updates only work when the Portfile appears in a commit. Maybe we need some sort of pipeline to regularly update the version.

arjunsalyan commented 1 year ago

Oh wow, thanks @jmroot . The errors are all gone, I will trigger a full database sync right away to speed up the cleanup process.

Total number of ports parsed:   633 
Ports successfully parsed:  633 
Ports failed:           0 
Up-to-date ports skipped:   29454
Dave-Allured commented 1 year ago

It looks like all the port info pages are now displaying correctly and not showing "deleted".

However, some ports are still not showing up in search results. qt5-qtbase is showing, but hdf5 and ncarg are not. Is there still some kind of synchronization problem?

arjunsalyan commented 1 year ago

The search index will catch up soon.

arjunsalyan commented 1 year ago

@Dave-Allured The search index seems to have synced. Please see if things are as expected now.

Dave-Allured commented 1 year ago

Yes, the search box is now finding all ports in question. Thanks @jmroot and @arjunsalyan for fast response.