Missing planning applications

[mvl22]

If you have become aware of a planning application, but it doesn't appear to be in Cyclescape, this is the place to let us know.

Why might a planning application be missing?

Our planning application system comes from a feed by PlanIt, a third party who we have been collaborating with. (They're doing a brilliant job - please consider donating!)

Therefore, if a planning application is missing, it is either because:

(a) PlanIt don't yet cover that Council (b) The Council hasn't yet put it on their website (c) It's only just appeared on the Council's website and it does take a day or two to be picked up (d) PlanIt hasn't managed to get it from the Council's website due to some technical problem (e) Or there could be a bug in Cyclescape

Where are planning applications shown in Cyclescape?

Go to My Cyclescape - it's the fifth tab on that page.

(i) The list of planning applications matches only the area(s) you are following. For instance, if you are subscribed to Cambridge, you won't see planning applications from Leeds.

(ii) However, the search box on the left that page will search for every planning application we have, regardless of area. You have to know the specific number.

Checklist before reporting issues

We definitely want to know if there's a problem, but please help us to help you by checking the followings points first:

1) Is the town/city/area concerned listed on the drop-down list on the left-hand side here? : http://www.planit.org.uk/

2) If you have an ID for the application, can you find it here? http://www.planit.org.uk/find/applics/ (If no, report the problem on this page, and ignore the remaining questions below.)

3) Has it been on the Council's website for more than a few days? Sometimes councils are a bit slow to put them online. There is then a 1-2 day delay before PlanIt picks them up, and for us to pick it up from PlanIt.

4) Have you tried a search (using the box on the left on this page) for the specific ID, if you know it?

5) Please make sure you specify the application ID if you know it.

May be this checklist will have solved the problem - but if not, please do report it here and let us know your answers to those questions.

[roxannedebeaux]

I have a planning application that isn't showing up. a) plan it covers this council b) the council has loaded it c) & d) it's been picked up by plan it

i) I have checked my areas and the area of this application is covered by my selected area on the map. ii) I've used the number and this application is not showing up.

This happens with at least one planning application every week.

You can see the application here: http://www.planit.org.uk/find/applics/#

[aspeakman]

To investigate any potential issues we need at least the reference ID of one application and the name of the local authority

[roxannedebeaux]

The details were all provided in the link

South Cambridgeshire S/2010/17/OL 08/06/2017 Station Road Garage, Station Yard, Station Road, Histon, Cambridge, CB24 9LF Erection of 32 flats including 4 live-work units with access road, parking an...

[aspeakman]

The PlanIt link you provided is not working - here is the correct version: http://www.planit.org.uk/planapplic/SouthCambridgeshire/S/2010/17/OL/

This application is dated 08/06/2017 and was only picked up by the PlanIt system this morning (12/06/2017). I am not sure how often Cyclescape updates, but I would allow a minimum of 24 hours before it could then be expected to appear on CycleScape

[nikolai-b]

Cyclescape updates at 1am every morning https://github.com/cyclestreets/cyclescape/blob/staging/config/schedule.rb#L42

[roxannedebeaux]

Another one hasn't appeared. I tried yesterday and again today so there has been time for it to be updated. This is a City Council application and I don't get problems as often with these.

17/0970/FUL https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=documents&keyVal=OQTLPDDXKJK00

[aspeakman]

Hi this application is dated 1st June - but was not picked up by PlanIt until 18th June 3 pm. At the moment I cannot tell you if that delay is unreasonable or was when the application appeared on the Cambridge website. However this means that the earliest it could have been picked up by Cyclescape would be 1am 19th June. The details are here http://www.planit.org.uk/planapplic/Cambridge/17/0970/FUL/

[roxannedebeaux]

Hmm strange. I accessed it on the City Council state on Monday Morning. I guess there are just always going to be some that are problematic and will likely fall through the cracks.

Is there anyway to push through a manual upload?

Perhaps this is a feature to think about for the future.

[roxannedebeaux]

Something seems to be happening with the Cambridge City Council planning applications this week. They were on the application list yesterday so were definitely loaded into their system. But 24 hours after that list was pushed (and therefore likely to have been at least 3 days since most were loaded) they are still not appearing in Cyclescape. I've never had so many applications missing.

There may be even more than these ones missing but here is a starting list: https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=relatedCases&keyVal=OUCXP6DX0AS00 https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=summary&keyVal=OU44CIDXM2Q00 https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=documents&keyVal=OUBCNWDX0AS00 https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=documents&keyVal=OUDEVWDX0BI00 https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=summary&keyVal=OUF4DKDX06P00

[mvl22]

Thanks for reporting this. I've checked, and all of them are not in PlanIt.

https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=relatedCases&keyVal=OUCXP6DX0AS00

14/0651/NMA1 Not in PlanIt

https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=summary&keyVal=OU44CIDXM2Q00

17/1372/FUL Not in PlanIt

https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=documents&keyVal=OUBCNWDX0AS00

17/1391/S73 Not in PlanIt

https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=documents&keyVal=OUDEVWDX0BI00

17/0008/COND22 Not in PlanIt

https://idox.cambridge.gov.uk/online-applications/applicationDetails.do?activeTab=summary&keyVal=OUF4DKDX06P00

17/1414/CTY Not in PlanIt

[aspeakman]

I have found some delays in updating the latest ids on the system which I am investigating. The site was scraped automatically yesterday night (15/08/17 - see below) but the previous occasion this happened was 10/08/17.

Four of the IDs listed worked there way into PlanIt yesterday (15/08/17 22:00) night as follows: http://www.planit.org.uk/planapplic/Cambridge/17/1372/FUL/ http://www.planit.org.uk/planapplic/Cambridge/17/1391/S73/ http://www.planit.org.uk/planapplic/Cambridge/17/0008/COND22/ http://www.planit.org.uk/planapplic/Cambridge/17/1414/CTY/

The other id (14/0651/NMA1) is known to the system and the detail will probably be gathered tomorrow.

[roxannedebeaux]

Is there any schedule for scraping that I could plan my activity around? It's the unreliability that causes the extra work. I can't be checking everyday so I'm doing it all manually. If I know what day each week I should look then I can plan around that.

[aspeakman]

Sorry there is no regular schedule in the PlanIt system - the scrapers run in sequence all the time and give way to the next one if there is no more work to do or if they reach a time limit.

The issue with the missed activity seems to be because PlanIt UK coverage is creeping up to 100% and there is a lot more to do each day.

So to address your problem I have now changed the settings so that each scraper should more reliably prioritise the check for the most current planning identifiers when it does run.

[stevenmos]

Colchester Borough Council (CBC) planning applications appear on PlanIt, but only up to 28/5/2019. CBC has been switching over to a new (but in my opinion not at all improved) website for some time so this must have been when they changed the planning system. The new database seems to be here.

In the Planning Application search, Cyclescape doesn't show any of the CBC applications from PlanIt that I've tried.

is this the right place to raise this?

[aspeakman]

The new Colchester planning website at https://www.colchester.gov.uk/planning-search/ was one where I found the system too complex. to crack. There are only a few such sites and others where there are specific measures to prevent scraping (such as Captcha) The list of sites currently inaccessible in this way is here -> https://www.planit.org.uk/news/

However I was not aware of the DataShare database you link to, so I will have a look at using that

[stevenmos]

OK, thanks.

For example, the following seems to reach the correct database:

wget http://datashare.colchester.gov.uk/download/planning-and-licensing/planning-applications-2010-31-3-2020/april-2020/XML

[mvl22]

... or the same URL ending /CSV

[aspeakman]

Thanks for the info.

I can use these data (there is a JSON end point here) but at the moment sadly it will not allow discovery of any current applications

It looks like the April 2020 designation refers to the extraction date (1st April) and the most recent data are those applications decided in the month of March - meaning the latest applications in the batch have a start date around 5th March.

[aspeakman]

The most recent Colchester applications (up to 5/3/2020) are now on the www.planit.org.uk site. It will take a couple of days for the backlog of historical applications to be fully processed.

[stevenmos]

Thanks Andrew. It looks like I'll have to pursue this locally. Do most authorities provide current (undecided) applications?

Will these soon start showing up in Cyclescape?

[mvl22]

We import nightly from PlanIt. So a day after it appears there.

[aspeakman]

Yes. Most authorities do provide a date search encompassing both decided and undecided.

Colchester do provide weekly lists of received applications - but only via the new interface that I could not manage to scrape

[stevenmos]

Thanks.

@mvl22, I searched Cyclescape for the some of the applications (e.g. below) that are on PlanIt but none that I've checked have worked. Even if there's a backlog, shouldn't the earlier ones have already been there because PlanIt had stopped at 28/5/19?

193143 | 23/12/2019, 102692 | 30/12/2010, RM/COL/00/1982 | 28/12/2000

[stevenmos]

Does anyone know why the CBC planning applications that are on PlanIt don't show up on Cyclescape?

e.g. a search for LB/COL/00/0142, or 200534 on Cyclescape does not return either.

Whereas, Reading's 200534 is shown

[aspeakman]

The Colchester LB/COL/00/0142 is quite old (last updated 2018) but the other one was scraped in April so probably should be there

Andrew

On Mon, 11 May 2020 at 12:58, Steven Moseley notifications@github.com wrote:

Does anyone know why the CBC planning applications that are on PlanIt don't show up on Cyclescape?

e.g. a search for LB/COL/00/0142 https://www.planit.org.uk/planapplic/Colchester/LB/COL/00/0142/, or 200534 https://www.planit.org.uk/planapplic/Colchester/200534/ on Cyclescape does not return either.

Whereas, Reading's 200534 https://www.planit.org.uk/planapplic/Reading/200534/ is shown https://www.cyclescape.org/planning_applications/uid/reading/200534

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cyclestreets/cyclescape/issues/715#issuecomment-626657560, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMOGTXWT6WLJRRDUEU6CHLRQ7R6HANCNFSM4DOBWFSA .

-- Andrew Speakman andrew@speakman.org.uk

[smsm1]

I'm trying to find SCC/0090/22IP in Cyclescape, however it's not showing up. SCC: http://suffolk.planning-register.co.uk/Planning/Display?applicationNumber=SCC%2F0090%2F22IP Planit: https://www.planit.org.uk/planapplic/Suffolk/SCC/0090/22IP/

I've not seen any recent planning applications in my area (Ipswich). Most are from March or earlier.

[mvl22]

I've not seen any recent planning applications in my area (Ipswich). Most are from March or earlier.

Can you confirm if it exists on www.planit.org.uk?

[smsm1]

The planning application does exist on planit.org.uk: https://www.planit.org.uk/planapplic/Suffolk/SCC/0090/22IP/

With the changes above I'm now seeing recent planning applications for the the borough level councils, however not seeing the application above at the county level. Does Suffolk need to be added as a local authority for planning somewhere?

[aspeakman]

By default counties show the applications for the county and enclosed districts. So often the county ones do not show up as they get outnumbered by more recent district ones.

To view the county ones only. Select Modify then Exclude sub-area applications

See https://www.planit.org.uk/planarea/Suffolk/

Andrew

On Thu, 8 Sep 2022, 22:06 Shaun McDonald, @.***> wrote:

The planning application does exist on planit.org.uk: https://www.planit.org.uk/planapplic/Suffolk/SCC/0090/22IP/

With the changes above I'm now seeing recent planning applications for the the borough level councils, however not seeing the application above at the county level. Does Suffolk need to be added as a local authority for planning somewhere?

— Reply to this email directly, view it on GitHub https://github.com/cyclestreets/cyclescape/issues/715#issuecomment-1241232948, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMOGTQN7LLIADIURTJJFHTV5JILRANCNFSM4DOBWFSA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[aspeakman]

Yes the link is in Shaun's original email

Andrew

On Tue, 6 Sep 2022, 13:13 Martin, @.***> wrote:

I've not seen any recent planning applications in my area (Ipswich). Most are from March or earlier.

Can you confirm if it exists on www.planit.org.uk?

— Reply to this email directly, view it on GitHub https://github.com/cyclestreets/cyclescape/issues/715#issuecomment-1238064617, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMOGTWISFLCC72HVDSYWB3V44YN3ANCNFSM4DOBWFSA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[mvl22]

https://www.planit.org.uk/planapplic/SouthCambs/23/01288/LBC/

doesn't seem to have come through. @nikolai-b can you check?

[nikolai-b]

That planning app has a start date of "2023-04-04" which is the only field date field I can filter on. If it didn't appear in PlantIt for till more than 14 days after that then it will be missed by us. If we could filter / sort on when it was changed in PlanIt then the system could be more robust as I'd just ask for the things changed in the past 2 days.

I've added it manually now.