search

relaton / relaton-3gpp

MIT License
2 stars 0 forks source link

(URGENT) `ftp://www.3gpp.org` no longer has bibliographic database #18

Closed ronaldtse closed 2 months ago

ronaldtse commented 9 months ago

From @andrew2net :

looks like 3GPP doesn't have ftp://www.3gpp.org/Information/Databases/Spec_Status/ folder on their FTP server anymore. Could you ask them how to get updates now?

For a very long time, 3GPP provided bibliographic entries through ftp://www.3gpp.org/ in form of the status_smg_3GPP.mdb Access database file.

Unfortunately, last week the 3GPP FTP site was updated, and that file no longer exists. This means that the IETF BibXML service (and hence RFCs and Internet-Drafts) would no longer be able to cite 3GPP publications moving forward.

ronaldtse commented 9 months ago

I have contacted @haudiobe to see if he would be able to help contact 3GPP. Thank you @haudiobe in advance!

andrew2net commented 7 months ago

@ronaldtse the new data source has duplicates like:

03.20ext;Security-related fetwork functions (Ext);https://www.3gpp.org/ftp/Specs/archive/03_series/03.20ext/0320ext-300.zip;3.0.0;Jan  1 1995 12:00AM;1;Nguyen Ngoc;Sebastien;Orange;S3;R6, R6, CP;Phase 1 extension;GSM_PH1;;;;;SA#25;Oct  1 1994 12:00AM;
03.20ext;Security-related fetwork functions (Ext);https://www.3gpp.org/ftp/Specs/archive/03_series/03.20ext/0320ext-300.zip;3.0.0;Sep  1 2003 12:00AM;1;Nguyen Ngoc;Sebastien;Orange;S3;R6, R6, CP;Phase 1 extension;GSM_PH1;;;;;SA#25;Oct  1 1994 12:00AM;

the only difference is date, Jan 1 1995 vs Sep 1 2003. Should we drop of the old docs and save the most recent only? Or should we add a year to ID to distinct them?

UPD There are documents that have different contributors:

37.885;Study on evaluation methodology of new Vehicle-to-Everything (V2X) use cases for LTE and NR;https://www.3gpp.org/ftp//Specs/archive/37_series/37.885/37885-001.zip;0.0.1;Mar 16 2018  8:55AM;0;Gulati;Kapil;Qualcomm UK Ltd;R1; ;Release 15;GSM_Release_15;3G4G5G_Rel-15;SA#76;SA#78;SA#83;;Jun  1 2016 12:00AM;Jun  7 2019  5:30PM
37.885;Study on evaluation methodology of new Vehicle-to-Everything (V2X) use cases for LTE and NR;https://www.3gpp.org/ftp//Specs/archive/37_series/37.885/37885-001.zip;0.0.1;Mar 16 2018  8:55AM;0;Khoryaev;Alexey;Intel Corporation (UK) Ltd;R1; ;Release 15;GSM_Release_15;3G4G5G_Rel-15;SA#76;SA#78;SA#83;;Jun  1 2016 12:00AM;Jun  7 2019  5:30PM
37.885;Study on evaluation methodology of new Vehicle-to-Everything (V2X) use cases for LTE and NR;https://www.3gpp.org/ftp//Specs/archive/37_series/37.885/37885-001.zip;0.0.1;Mar 16 2018  8:55AM;0;Seo;Hanbyul;LG Electronics Inc.;R1; ;Release 15;GSM_Release_15;3G4G5G_Rel-15;SA#76;SA#78;SA#83;;Jun  1 2016 12:00AM;Jun  7 2019  5:30PM
37.885;Study on evaluation methodology of new Vehicle-to-Everything (V2X) use cases for LTE and NR;https://www.3gpp.org/ftp//Specs/archive/37_series/37.885/37885-001.zip;0.0.1;Mar 16 2018  8:55AM;0;Yasukawa;Shinpei;DOCOMO Communications Laboratories Europe GmbH;R1; ;Release 15;GSM_Release_15;3G4G5G_Rel-15;SA#76;SA#78;SA#83;;Jun  1 2016 12:00AM;Jun  7 2019  5:30PM

Maybe we should merge such the documents. What do you think?

ronaldtse commented 7 months ago

@andrew2net are these the only problematic ones? Can you list them all out so we can ask 3GPP all at once?

andrew2net commented 7 months ago

@ronaldtse there are 1624 duplication groups. They are grouped by "Spec number" and "Version". Each group has from 2 to 5 duplications. See all list in the attachment: 3gpp_dups.txt

UPD I think the duplicates are needed because there is no way to have more than one date or contributor in a single table without having extra columns. Contributors can be merged without any problem. But for dates we need to decide what types they are. Maybe we can assign types chronologically? For example first date released, second published, and so on. The CSV dataset also doesn't have documents' statuses. Maybe it possible to get status from document's chronology?

ronaldtse commented 6 months ago

Here are the two questions asked to 3GPP:

  1. Some CSV lines have identical content but different dates
  2. Some CSV lines have identical content but different contributors

Answer from Dongwook Kim of 3GPP:

  1. The difference is that whether the version has a corresponding transposed standards by the organizing partner SDOs or not. The version in 1995 does not have a transposed version, whereas the version in 2003 does.
  2. Yes your understanding is correct. There are multiple rapporteurs for 37.885.
ronaldtse commented 6 months ago

Question:

Regarding question 1: so this is the same document, that was originally published in 1995, and then it was republished with a partner SDO in 2005? Or can there be a flag to explicitly say that it was re-published at the partner SDO?

Answer from Dongwook Kim of 3GPP:

It is possible to create a flag, but I would prefer (for simplicity reasons) to have another column name “ETSI Transposition” that states the ETSI standard to which the 3GPP specification was transposed to. For the example below, it would be NULL for 1995, and something like “ETSI TS 103 020ext…”.

ronaldtse commented 6 months ago

Answer from Dongwook Kim:

Just to confirm, you are OK if it can be distinguished by “NULL” (no transposition) or any other values (yes transposition), right?

If so, I will update the excel accordingly from this March plenary 😊

@andrew2net do we have sufficient information? Thanks!

haudiobe commented 6 months ago

I am forwarding the question to ETSI MCC if still needed

ronaldtse commented 6 months ago

Thank you @haudiobe - I think we have everything sorted out with the help of Dongwook. Thanks!

ronaldtse commented 6 months ago

The data is at:

Future:

andrew2net commented 2 months ago

@ronaldtse so what does the "transposed" date mean in the Relaton context? Should we save the date as a specific type? We have following date types:

  <define name="BibliographicDateType">
    <choice>
      <value>published</value>
      <value>accessed</value>
      <value>created</value>
      <value>implemented</value>
      <value>obsoleted</value>
      <value>confirmed</value>
      <value>updated</value>
      <value>corrected</value>
      <value>issued</value>
      <value>transmitted</value>
      <value>copied</value>
      <value>unchanged</value>
      <value>circulated</value>
      <value>adapted</value>
      <value>vote-started</value>
      <value>vote-ended</value>
      <value>announced</value>
      <value>stable-until</value>
    </choice>
  </define>

Which one should we use?

UPD There is no anything like “ETSI Transposition” in the dataset

Future:

ronaldtse commented 2 months ago

I'm not sure if we have any corresponding relation. What they mean by "transpose" is that the document has been sent to be published in a new place.

So instead of a date, it should be a standard relationship of:

andrew2net commented 2 months ago

@ronaldtse we don't have any "transpose" flag in the dataset. Can we infer that the earlier published document is a main publication and all others are adoptedAs relations?

ronaldtse commented 2 months ago

Yes. That is the correct understanding. The first item that is published is the real one. The subsequent items that are “transposed” means they have been published at other venues with the same content and unmodified, so these are adoptions.

andrew2net commented 2 months ago

I found duplications missing a link and a date in one of the rows. It looks like we should drop the row with missing data. I just put these cases here for information. 

...
22.135,Multicall; Service description; Stage 1,,4.2.0,,1,Miettinen,Natalia,NOKIA UK R&D Ltd,S1, ,Release 4,GSM_Release_4,3G_Rel-4,SA-#11,SA-#11,SA-#11,SA#65,Aug  1 1998 12:00AM,Jun 21 2001 12:00AM
22.135,Multicall; Service description; Stage 1,https://www.3gpp.org/ftp/Specs/archive/22_series/22.135/22135-420.zip,4.2.0,Dec 18 2002 12:00AM,1,Miettinen,Natalia,NOKIA UK R&D Ltd,S1, ,Release 4,GSM_Release_4,3G_Rel-4,SA-#11,SA-#11,SA-#11,SA#65,Aug  1 1998 12:00AM,Jun 21 2001 12:00AM
...
23.078,Customised Applications for Mobile network Enhanced Logic (CAMEL) Phase 4; Stage 2,,4.5.0,,1,Wiehe,Ulrich,Nokia Solutions and Networks (shanghai) Ltd.,C4, ,Release 4,GSM_Release_4,3G_Rel-4,SA-#11,SA-#11,SA-#11,SA#65,Aug  1 1998 12:00AM,Jun 21 2001 12:00AM
23.078,Customised Applications for Mobile network Enhanced Logic (CAMEL) Phase 4; Stage 2,https://www.3gpp.org/ftp/Specs/archive/23_series/23.078/23078-450.zip,4.5.0,Jun 17 2002 12:00AM,1,Wiehe,Ulrich,Nokia Solutions and Networks (shanghai) Ltd.,C4, ,Release 4,GSM_Release_4,3G_Rel-4,SA-#11,SA-#11,SA-#11,SA#65,Aug  1 1998 12:00AM,Jun 21 2001 12:00AM
...
23.882,3GPP system architecture evolution (SAE): Report on technical options and conclusions,,0.3.0,,0,Pudney,Chris,Vodafone Group Plc,S2, ,Release 7,GSM_Release_7,3G_Rel-7,SA#29,SA#33,SA#35,SA#65,Oct  6 2003 12:00AM,Mar 13 2008  1:00PM
23.882,3GPP system architecture evolution (SAE): Report on technical options and conclusions,https://www.3gpp.org/ftp/Specs/archive/23_series/23.882/23882-030.zip,0.3.0,Jul 26 2005 12:00AM,0,Pudney,Chris,Vodafone Group Plc,S2, ,Release 7,GSM_Release_7,3G_Rel-7,SA#29,SA#33,SA#35,SA#65,Oct  6 2003 12:00AM,Mar 13 2008  1:00PM
...
24.228,Signalling flows for the IP multimedia call control based on Session Initiation Protocol (SIP) and Session Description Protocol (SDP); Stage 3,,1.0.0,,1,Mayer,Georg,"HuaWei Technologies Co., Ltd",C1, ,Release 5,GSM_Release_5,3G_Rel-5,SA-#15,SA-#15,SA-#15,SA#65,May  1 2000 12:00AM,Sep 12 2002  5:00PM
24.228,Signalling flows for the IP multimedia call control based on Session Initiation Protocol (SIP) and Session Description Protocol (SDP); Stage 3,https://www.3gpp.org/ftp/Specs/archive/24_series/24.228/24228-100.zip,1.0.0,Jun 26 2001 12:00AM,1,Mayer,Georg,"HuaWei Technologies Co., Ltd",C1, ,Release 5,GSM_Release_5,3G_Rel-5,SA-#15,SA-#15,SA-#15,SA#65,May  1 2000 12:00AM,Sep 12 2002  5:00PM
...
24.605,Conference (CONF) using IP Multimedia (IM) Core Network (CN) subsystem; Protocol specification,,2.6.0,,1,Ji,Mengdi,Huawei Technologies France,C1, ,Release 8,GSM_Release_8,3G_Rel-8,SA#38,SA#40,SA#42,,Jan 23 2006 12:00AM,Mar 12 2009  1:00PM
24.605,Conference (CONF) using IP Multimedia (IM) Core Network (CN) subsystem; Protocol specification,https://www.3gpp.org/ftp/Specs/archive/24_series/24.605/24605-260.zip,2.6.0,Feb  8 2008 12:00AM,1,Ji,Mengdi,Huawei Technologies France,C1, ,Release 8,GSM_Release_8,3G_Rel-8,SA#38,SA#40,SA#42,,Jan 23 2006 12:00AM,Mar 12 2009  1:00PM
...
25.331,Radio Resource Control (RRC); Protocol specification,,6.0.0,,1,Shreevastav,Ritesh,Ericsson India Private Limited,RP,R2,Release 6,GSM_Release_6,3G_Rel-6,SA#26,SA#26,SA#27,SA#65,Mar 28 2000 12:00AM,Sep 28 2005  1:00PM
25.331,Radio Resource Control (RRC); Protocol specification,https://www.3gpp.org/ftp/Specs/archive/25_series/25.331/25331-600.zip,6.0.0,Jan 12 2004 12:00AM,1,Shreevastav,Ritesh,Ericsson India Private Limited,RP,R2,Release 6,GSM_Release_6,3G_Rel-6,SA#26,SA#26,SA#27,SA#65,Mar 28 2000 12:00AM,Sep 28 2005  1:00PM
...
25.809,7.68 Mcps TDD option: Physical layer,,0.5.2,,0,Beale,Martin,Waiting for new Company,RP,R1,Release 7,GSM_Release_7,3G_Rel-7,SA#29,SA#33,SA#35,SA#65,Oct  6 2003 12:00AM,Mar 13 2008  1:00PM
25.809,7.68 Mcps TDD option: Physical layer,https://www.3gpp.org/ftp/Specs/archive/25_series/25.809/25809-052.zip,0.5.2,Nov 21 2005 12:00AM,0,Beale,Martin,Waiting for new Company,RP,R1,Release 7,GSM_Release_7,3G_Rel-7,SA#29,SA#33,SA#35,SA#65,Oct  6 2003 12:00AM,Mar 13 2008  1:00PM
...
25.880,Re-arrangement of Iub transport bearers,,0.3.0,,0,Hautala,Jari,Nokia Corporation,R3, ,Release 5,GSM_Release_5,3G_Rel-5,SA-#15,SA-#15,SA-#15,SA#65,May  1 2000 12:00AM,Sep 12 2002  5:00PM
25.880,Re-arrangement of Iub transport bearers,https://www.3gpp.org/ftp/Specs/archive/25_series/25.880/25880-030.zip,0.3.0,Jan 11 2002 12:00AM,0,Hautala,Jari,Nokia Corporation,R3, ,Release 5,GSM_Release_5,3G_Rel-5,SA-#15,SA-#15,SA-#15,SA#65,May  1 2000 12:00AM,Sep 12 2002  5:00PM
...
29.230,Diameter applications; 3GPP specific codes and identifiers,,12.0.0,,1,Wiehe,Ulrich,Nokia Solutions and Networks (shanghai) Ltd.,C4, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
29.230,Diameter applications; 3GPP specific codes and identifiers,https://www.3gpp.org/ftp/Specs/archive/29_series/29.230/29230-c00.zip,12.0.0,Mar 13 2013 12:00AM,1,Wiehe,Ulrich,Nokia Solutions and Networks (shanghai) Ltd.,C4, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
...
29.817,Study on eXtensible Markup Language (XML) based access of the Application Function (AF) to the Policy and Charging Rules Function (PCRF),,1.0.0,,0,Brinkmann,Horst,Alcatel-Lucent Deutschland AG,C3, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
29.817,Study on eXtensible Markup Language (XML) based access of the Application Function (AF) to the Policy and Charging Rules Function (PCRF),https://www.3gpp.org/ftp/Specs/archive/29_series/29.817/29817-100.zip,1.0.0,Dec 16 2013 12:00AM,0,Brinkmann,Horst,Alcatel-Lucent Deutschland AG,C3, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
...
31.103,Characteristics of the IP Multimedia Services Identity Module (ISIM) application,,7.0.0,,1,Phan,Ly Thanh,Gemalto N.V.,C6, ,Release 7,GSM_Release_7,3G_Rel-7,SA#29,SA#33,SA#35,SA#65,Oct  6 2003 12:00AM,Mar 13 2008  1:00PM
31.103,Characteristics of the IP Multimedia Services Identity Module (ISIM) application,https://www.3gpp.org/ftp/Specs/archive/31_series/31.103/31103-700.zip,7.0.0,Jun 12 2006 12:00AM,1,Phan,Ly Thanh,Gemalto N.V.,C6, ,Release 7,GSM_Release_7,3G_Rel-7,SA#29,SA#33,SA#35,SA#65,Oct  6 2003 12:00AM,Mar 13 2008  1:00PM
...
32.766,Telecommunication management; Evolved Universal Terrestrial Radio Access Network (E-UTRAN) Network Resource Model (NRM) Integration Reference Point (IRP); Solution Set (SS) definitions,,10.8.0,,1,Zou,Lan,"Huawei Technologies (UK) Co., Ltd.",S5, ,Release 10,GSM_Release_10,3G4G_Rel-10,SA#47,SA#49,SA#51,,Jan 20 2009 12:00AM,Jun  8 2011  5:30PM
32.766,Telecommunication management; Evolved Universal Terrestrial Radio Access Network (E-UTRAN) Network Resource Model (NRM) Integration Reference Point (IRP); Solution Set (SS) definitions,https://www.3gpp.org/ftp/Specs/archive/32_series/32.766/32766-a80.zip,10.8.0,Mar 17 2014 12:00AM,1,Zou,Lan,"Huawei Technologies (UK) Co., Ltd.",S5, ,Release 10,GSM_Release_10,3G4G_Rel-10,SA#47,SA#49,SA#51,,Jan 20 2009 12:00AM,Jun  8 2011  5:30PM
...
33.21U,Security requirements,,3.0.0,,1,Bergmann,Ansgar,Hillebrand Consulting Engineers GmbH,, ,UMTS,,,SMG-#28,SMG-#28,SMG-#28,SA#28,Aug 18 1994 12:00AM,Feb 12 1999 12:00AM
33.21U,Security requirements,https://www.3gpp.org/ftp/Specs/archive/33_series/33.21U/3321U-300.zip,3.0.0,Feb 21 1999 12:00AM,1,Bergmann,Ansgar,Hillebrand Consulting Engineers GmbH,, ,UMTS,,,SMG-#28,SMG-#28,SMG-#28,SA#28,Aug 18 1994 12:00AM,Feb 12 1999 12:00AM
...
33.816,Feasibility study on LTE relay node security,,2.0.0,,0,Escott,Adrian,Qualcomm Germany GmbH,S3, ,Release 11,GSM_Release_11,3G4G_Rel-11,SA#53,SA#55,SA#57,,Jan 22 2010 12:00AM,Mar  6 2013  5:30PM
33.816,Feasibility study on LTE relay node security,https://www.3gpp.org/ftp/Specs/archive/33_series/33.816/33816-200.zip,2.0.0,Mar 28 2011  9:44AM,0,Escott,Adrian,Qualcomm Germany GmbH,S3, ,Release 11,GSM_Release_11,3G4G_Rel-11,SA#53,SA#55,SA#57,,Jan 22 2010 12:00AM,Mar  6 2013  5:30PM
...
36.836,Evolved Universal Terrestrial Radio Access (E-UTRA); Study on mobile relay,,12.0.0,,0,Hu,Haijing,China Academy of Telecommunication Technology - (Datang Telecom Technology and Industry Group),R3, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
36.836,Evolved Universal Terrestrial Radio Access (E-UTRA); Study on mobile relay,https://www.3gpp.org/ftp/Specs/archive/36_series/36.836/36836-c00.zip,12.0.0,Jun 27 2014 12:00AM,0,Hu,Haijing,China Academy of Telecommunication Technology - (Datang Telecom Technology and Industry Group),R3, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
...
36.851,Inter-band carrier aggregation,,0.12.0,,0,Khan,Asif Ali,Ericsson Telefonaktiebolaget LM,R4, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
36.851,Inter-band carrier aggregation,https://www.3gpp.org/ftp/Specs/archive/36_series/36.851/36851-0c0.zip,0.12.0,Sep 29 2014 12:00AM,0,Khan,Asif Ali,Ericsson Telefonaktiebolaget LM,R4, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
...
36.851,Inter-band carrier aggregation,,0.4.0,,0,Khan,Asif Ali,Ericsson Telefonaktiebolaget LM,R4, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
36.851,Inter-band carrier aggregation,https://www.3gpp.org/ftp/Specs/archive/36_series/36.851/36851-040.zip,0.4.0,Apr 25 2013 12:00AM,0,Khan,Asif Ali,Ericsson Telefonaktiebolaget LM,R4, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
...
37.902,Measurements of User Equipment (UE) radio performances for LTE/UMTS terminals; Total Radiated Power (TRP) and Total Radiated Sensitivity (TRS) test methodology,,12.0.0,,0,An,Xudong,CATR,R4, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
37.902,Measurements of User Equipment (UE) radio performances for LTE/UMTS terminals; Total Radiated Power (TRP) and Total Radiated Sensitivity (TRS) test methodology,https://www.3gpp.org/ftp/Specs/archive/37_series/37.902/37902-c00.zip,12.0.0,Oct  3 2014 12:00AM,0,An,Xudong,CATR,R4, ,Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
...
43.020,Security related network functions,,10.2.0,,1,Sahlin,Bengt,Ericsson Telefonaktiebolaget LM,S3,"RP, CP",Release 10,GSM_Release_10,3G4G_Rel-10,SA#47,SA#49,SA#51,,Jan 20 2009 12:00AM,Jun  8 2011  5:30PM
43.020,Security related network functions,https://www.3gpp.org/ftp/Specs/archive/43_series/43.020/43020-a20.zip,10.2.0,Jun 27 2014 12:00AM,1,Sahlin,Bengt,Ericsson Telefonaktiebolaget LM,S3,"RP, CP",Release 10,GSM_Release_10,3G4G_Rel-10,SA#47,SA#49,SA#51,,Jan 20 2009 12:00AM,Jun  8 2011  5:30PM
...
43.020,Security related network functions,,11.2.0,,1,Sahlin,Bengt,Ericsson Telefonaktiebolaget LM,S3,"RP, CP",Release 11,GSM_Release_11,3G4G_Rel-11,SA#53,SA#55,SA#57,,Jan 22 2010 12:00AM,Mar  6 2013  5:30PM
43.020,Security related network functions,https://www.3gpp.org/ftp/Specs/archive/43_series/43.020/43020-b20.zip,11.2.0,Jun 27 2014 12:00AM,1,Sahlin,Bengt,Ericsson Telefonaktiebolaget LM,S3,"RP, CP",Release 11,GSM_Release_11,3G4G_Rel-11,SA#53,SA#55,SA#57,,Jan 22 2010 12:00AM,Mar  6 2013  5:30PM
...
43.020,Security related network functions,,12.1.0,,1,Sahlin,Bengt,Ericsson Telefonaktiebolaget LM,S3,"RP, CP",Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
43.020,Security related network functions,https://www.3gpp.org/ftp/Specs/archive/43_series/43.020/43020-c10.zip,12.1.0,Jun 27 2014 12:00AM,1,Sahlin,Bengt,Ericsson Telefonaktiebolaget LM,S3,"RP, CP",Release 12,GSM_Release_12,3G4G_Rel-12,SA#59,SA#62,SA#65,,Jun 26 2011 12:00AM,Mar 13 2015  5:00PM
...
43.020,Security related network functions,,9.2.0,,1,Sahlin,Bengt,Ericsson Telefonaktiebolaget LM,S3,"RP, CP",Release 9,GSM_Release_9,3G_Rel-9,SA#42,SA#44,SA#46,,Mar  6 2008 12:00AM,Mar 25 2010  1:00PM
43.020,Security related network functions,https://www.3gpp.org/ftp/Specs/archive/43_series/43.020/43020-920.zip,9.2.0,Jun 27 2014 12:00AM,1,Sahlin,Bengt,Ericsson Telefonaktiebolaget LM,S3,"RP, CP",Release 9,GSM_Release_9,3G_Rel-9,SA#42,SA#44,SA#46,,Mar  6 2008 12:00AM,Mar 25 2010  1:00PM
...
51.010-1,Mobile Station (MS) conformance specification; Part 1: Conformance specification,,5.4.0,,1,Lascoux,Remi,"Sierra Wireless, S.A.",R5,"S1, CP",Release 5,GSM_Release_5,3G_Rel-5,SA-#15,SA-#15,SA-#15,SA#65,May  1 2000 12:00AM,Sep 12 2002  5:00PM
51.010-1,Mobile Station (MS) conformance specification; Part 1: Conformance specification,https://www.3gpp.org/ftp/Specs/archive/51_series/51.010-1/51010-1-540.zip,5.4.0,Jul 22 2003 12:00AM,1,Lascoux,Remi,"Sierra Wireless, S.A.",R5,"S1, CP",Release 5,GSM_Release_5,3G_Rel-5,SA-#15,SA-#15,SA-#15,SA#65,May  1 2000 12:00AM,Sep 12 2002  5:00PM
...
51.010-2,Mobile Station (MS) conformance specification; Part 2: Protocol Implementation Conformance Statement (PICS) proforma specification,,5.4.0,,1,Lascoux,Remi,"Sierra Wireless, S.A.",R5,"S1, CP",Release 5,GSM_Release_5,3G_Rel-5,SA-#15,SA-#15,SA-#15,SA#65,May  1 2000 12:00AM,Sep 12 2002  5:00PM
51.010-2,Mobile Station (MS) conformance specification; Part 2: Protocol Implementation Conformance Statement (PICS) proforma specification,https://www.3gpp.org/ftp/Specs/archive/51_series/51.010-2/51010-2-540.zip,5.4.0,Jul 17 2003 12:00AM,1,Lascoux,Remi,"Sierra Wireless, S.A.",R5,"S1, CP",Release 5,GSM_Release_5,3G_Rel-5,SA-#15,SA-#15,SA-#15,SA#65,May  1 2000 12:00AM,Sep 12 2002  5:00PM
...

This duplication has one document without date and another with incorrect link:

29.812,Study on SCC AS Restoration,https://www.3gpp.org/ftp/Specs/archive/29_series/29.812/29812-010.zip,0.1.0,,0,Du,Xiaoning,China Academy of Telecommunication Research of MIIT,C4, ,Release 13,GSM_Release_13,3G4G_Rel-13,SA#65,SA#68,SA#70,,Sep 30 2012 12:00AM,Mar 11 2016  5:00PM
29.812,Study on SCC AS Restoration,https://www.3gpp.org/ftpSpecs/archive/29_series/29.812/29812-010.zip,0.1.0,Jun 25 2015  6:18AM,0,Du,Xiaoning,China Academy of Telecommunication Research of MIIT,C4, ,Release 13,GSM_Release_13,3G4G_Rel-13,SA#65,SA#68,SA#70,,Sep 30 2012 12:00AM,Mar 11 2016  5:00PM

These duplications are completely identical

...
25.813,Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN); Radio interface protocol aspects,https://www.3gpp.org/ftp/Specs/archive/25_series/25.813/25813-030.zip,0.3.0,Jan 18 2006  9:35AM,0,SEBIRE,Benoist,Nokia Solutions and Networks GmbH & Co. KG,RP,R2,Release 7,GSM_Release_7,3G_Rel-7,SA#29,SA#33,SA#35,SA#65,Oct  6 2003 12:00AM,Mar 13 2008  1:00PM
25.813,Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN); Radio interface protocol aspects,https://www.3gpp.org/ftp/Specs/archive/25_series/25.813/25813-030.zip,0.3.0,Jan 18 2006 12:00AM,0,SEBIRE,Benoist,Nokia Solutions and Networks GmbH & Co. KG,RP,R2,Release 7,GSM_Release_7,3G_Rel-7,SA#29,SA#33,SA#35,SA#65,Oct  6 2003 12:00AM,Mar 13 2008  1:00PM
...
29.935,Study on User Data Convergence (UDC) data model,https://www.3gpp.org/ftp/Specs/archive/29_series/29.935/29935-010.zip,0.1.0,Jul 21 2010 10:12AM,0,Ficaccio,Mauro,Telecom Italia S.p.A.,C4, ,Release 10,GSM_Release_10,3G4G_Rel-10,SA#47,SA#49,SA#51,,Jan 20 2009 12:00AM,Jun  8 2011  5:30PM
29.935,Study on User Data Convergence (UDC) data model,https://www.3gpp.org/ftp/Specs/archive/29_series/29.935/29935-010.zip,0.1.0,Jul 21 2010 12:00AM,0,Ficaccio,Mauro,Telecom Italia S.p.A.,C4, ,Release 10,GSM_Release_10,3G4G_Rel-10,SA#47,SA#49,SA#51,,Jan 20 2009 12:00AM,Jun  8 2011  5:30PM
...
andrew2net commented 2 months ago

@ronaldtse the new dataset doesn't have editorialgroup, but our datamodel requires it. @opoudjis could we make editorialgroup optional in the BibDataExtensionType definition in relaton-3gpp.rng?

ronaldtse commented 2 months ago

@ronaldtse the new dataset doesn't have editorialgroup, but our datamodel requires it. @opoudjis could we make editorialgroup optional in the BibDataExtensionType definition in relaton-3gpp.rng?

Yes, thanks @andrew2net .

andrew2net commented 2 months ago

Parser for the new datase is implemented in v1.19.1