search

ckan / ckan

CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers catalog.data.gov, open.canada.ca/data, data.humdata.org among many other sites.
https://ckan.org/
Other
4.39k stars 1.98k forks source link

http://demo.ckan.org/ not on 2.9.x release #5788

Open duttonw opened 3 years ago

duttonw commented 3 years ago

CKAN version 2.8.2 https://demo.ckan.org/api/3/action/status_show

Since the demo ckan is meant to be showing off the latest and greatest it seems its a little out of date, 2.8.2 came out on Dec 12, 2018.

amercader commented 3 years ago

@pdelboca @rufuspollock anyone at Datopian has capacity to work on this (as it is hosted by Datopioan)?

rufuspollock commented 3 years ago

@amercader Yes, something I think we could look at early in the New Year (bit packed atm b4 xmas). @shubham-mahajan would prob lead on this.

shubham-mahajan commented 3 years ago

@amercader Can you assign the issue to me?

amercader commented 3 years ago

@shubham-mahajan @avdata99 @rufuspollock Demo is currently overrun by spam.

As part of the upgrade process perhaps we can add a simple job that resets the database every day (or at least deletes and purges all users and datasets)?

shubham-mahajan commented 3 years ago

@amercader Sure.

amercader commented 3 years ago

@shubham-mahajan thanks! Do you have a timeline for when you'll work on this? Even if still not updating would be good to clear the DB and all the spam with it.

shubham-mahajan commented 3 years ago

@amercader Cleaning of the DB is done, we will setup the job to clean the database.

We are working on deploying to 2.9 also, will get that done soon.

amercader commented 3 years ago

Looks like spam is back already :( We might have to make it read-only and disable web registration

duttonw commented 3 years ago

I think there is a recaptcha plugin somewhere to restrict user registration. Might also need to add to dataset/resource update.

Might also want to pause account use till email is verified (unsure if it emails)

Also put a robots.txt to block search engines from scraping it so that they can’t use it for seo etc.

duttonw commented 3 years ago

There is many plugins but I don't see them updated to support 2.9 nor ability to verify recaptcha server side. https://github.com/DataShades/ckanext-recaptcha https://github.com/DataShades/ckanext-textcaptcha https://github.com/Hoedic/ckanext-recaptcha

shubham-mahajan commented 3 years ago

@amercader Let me find some solution for that, what do you think about stopping registration by API?

And there are some extensions that are not supported by 2.9 yet, we are not adding that to current demo.ckan.org

Zharktas commented 3 years ago

Most spammers just fill forms, so disabling form registration should help. If the spammers actually use the api, there's really not much which can be done with open registration.

shubham-mahajan commented 3 years ago

@Zharktas But if registration is disabled, people will not be able to use any functionality in the portal.

amercader commented 3 years ago

@shubham-mahajan The site is flooded with spam again. Let's clean up the database again, put up some demo data and disable user registration for now. We can think of a workflow for registering trusted users later on.

shubham-mahajan commented 3 years ago

@amercader Disable registration via both API and Web?

This sounds good to me for now.

duttonw commented 3 years ago

Maybe just web and see if it gets flooded again. Could also create a non-admin user to show login and profile

shubham-mahajan commented 3 years ago

@amercader This is done

loleg commented 2 years ago

I have noticed a couple of other CKAN instances facing similar problems. Is there a feature or extension that allows public flagging of datasets for moderation? I suppose comments could be used in this way. The legacy moderated edits plugin also did something to this effect. Possibly related: #1139