search

orientechnologies / orientdb

OrientDB is the most versatile DBMS supporting Graph, Document, Reactive, Full-Text and Geospatial models in one Multi-Model product. OrientDB can run distributed (Multi-Master), supports SQL, ACID Transactions, Full-Text indexing and Reactive Queries.
https://orientdb.dev
Apache License 2.0
4.75k stars 871 forks source link

Class creation takes too much time #4649

Closed iw2rmb closed 8 years ago

iw2rmb commented 9 years ago

Hello! Having perfomance issue with classes creation. No indexes, approx 300 subclasses over one class takes several hours to create via orientjs. How can I speed that up?

nagarajasr commented 9 years ago

which version of orientdb?

iw2rmb commented 9 years ago

v2.0.12

nagarajasr commented 9 years ago

does the orientdb console also experience similar performance issue or does it happen only through orientjs driver?

iw2rmb commented 9 years ago

It takes 10s of seconds or even minute to create next one over console too.

iw2rmb commented 9 years ago

Any guess? At last, orientdb web-client takes up to minutes while opening schema of database if last one has hundreds of classes.

nagarajasr commented 9 years ago

@iw2rmb any chance you could provide a script to reproduce the problem?

iw2rmb commented 9 years ago 
create class Lemma
create class Bulgarian extends Lemma
create class English extends Lemma
create class French extends Lemma
create class Castilian extends Lemma
create class Russian extends Lemma
create class Dutch extends Lemma
create class Czech extends Lemma
create class Swedish extends Lemma
create class Slovenian extends Lemma
create class Polish extends Lemma
create class Portuguese extends Lemma
create class Bokm��l extends Lemma
create class Basque extends Lemma
create class Slovak extends Lemma
create class Estonian extends Lemma
create class Finnish extends Lemma
create class Italian extends Lemma
create class German extends Lemma
create class Hungarian extends Lemma
create class Danish extends Lemma
create class English-UnitedStates- extends Lemma
create class Greek extends Lemma
create class Hebrew extends Lemma
create class Mandarin-simplified- extends Lemma
create class Neapolitan extends Lemma
create class Marathi extends Lemma
create class Lombard extends Lemma
create class Japanese extends Lemma
create class Esperanto extends Lemma
create class Piedmontese extends Lemma
create class Venetian extends Lemma
create class Ukrainian extends Lemma
create class Catalan extends Lemma
create class Ewe extends Lemma
create class Akan extends Lemma
create class Arabic extends Lemma
create class Hausa-Ajami- extends Lemma
create class Hausa-Latin- extends Lemma
create class Armenian extends Lemma
create class Georgian extends Lemma
create class Afrikaans extends Lemma
create class French-Canada- extends Lemma
create class French-Switzerland- extends Lemma
create class Serbian-Cyrillicscript- extends Lemma
create class Serbian-Latinscript- extends Lemma
create class Thai extends Lemma
create class Ido extends Lemma
create class Romanian extends Lemma
create class K��lsch extends Lemma
create class Mandarin-traditional- extends Lemma
create class English-UnitedKingdom- extends Lemma
create class Turkish extends Lemma
create class Korean extends Lemma
create class Khmer extends Lemma
create class Sicilian extends Lemma
create class Aragonese extends Lemma
create class Persian extends Lemma
create class Latvian extends Lemma
create class Vietnamese extends Lemma
create class Swahili extends Lemma
create class Bavarian extends Lemma
create class Belarusian extends Lemma
create class Kashubian extends Lemma
create class Croatian extends Lemma
create class Lithuanian extends Lemma
create class Cantonese extends Lemma
create class Tajik extends Lemma
create class Welsh extends Lemma
create class Bambara extends Lemma
create class Portuguese-Brazil- extends Lemma
create class Portuguese-Portugal- extends Lemma
create class Telugu extends Lemma
create class Gilaki extends Lemma
create class EasternMari extends Lemma
create class WesternMari extends Lemma
create class Gujarati extends Lemma
create class Hindi extends Lemma
create class Tamil extends Lemma
create class Indonesian extends Lemma
create class Irish extends Lemma
create class BelgianFrench extends Lemma
create class Limburgish extends Lemma
create class AlemannicGerman extends Lemma
create class Marwari extends Lemma
create class Sranan extends Lemma
create class German-Austria- extends Lemma
create class German-Switzerland- extends Lemma
create class AncientGreek extends Lemma
create class MycenaeanGreek extends Lemma
create class CappadocianGreek extends Lemma
create class Tosk extends Lemma
create class Gheg extends Lemma
create class Arvanitic extends Lemma
create class Arb��resh��Albanian extends Lemma
create class Kannada extends Lemma
create class Breton extends Lemma
create class Frisian extends Lemma
create class Navajo extends Lemma
create class Bengali extends Lemma
create class Urdu extends Lemma
create class French-France- extends Lemma
create class Nepali extends Lemma
create class Igbo extends Lemma
create class Ladin extends Lemma
create class Kazakh extends Lemma
create class Walloon extends Lemma
create class Icelandic extends Lemma
create class Tatar extends Lemma
create class Asturian extends Lemma
create class Sindhi-Arabicscript- extends Lemma
create class Sindhi-Gurumuki- extends Lemma
create class Yoruba extends Lemma
create class Galician extends Lemma
create class Nynorsk extends Lemma
create class Lingala extends Lemma
create class Xhosa extends Lemma
create class UpperSorbian extends Lemma
create class LowerSorbian extends Lemma
create class Tagalog extends Lemma
create class Maltese extends Lemma
create class Cebuano extends Lemma
create class Macedonian extends Lemma
create class Interlingua extends Lemma
create class Interlingue extends Lemma
create class Faroese extends Lemma
create class Zulu extends Lemma
create class Sanskrit extends Lemma
create class NepalBhasa extends Lemma
create class Malayalam extends Lemma
create class Flemish extends Lemma
create class Chickasaw extends Lemma
create class Corsican extends Lemma
create class Volap��k extends Lemma
create class Malay extends Lemma
create class Kabyle extends Lemma
create class Manx extends Lemma
create class Ossetian extends Lemma
create class CrimeanTatar extends Lemma
create class Bashkir extends Lemma
create class Cherokee extends Lemma
create class Kyrgyz extends Lemma
create class Kiribati extends Lemma
create class Javanese extends Lemma
create class Extremaduran extends Lemma
create class Tetum extends Lemma
create class Mirandese extends Lemma
create class Emiliano_Romagnolo extends Lemma
create class Wolof extends Lemma
create class Somali extends Lemma
create class Latin extends Lemma
create class Novial extends Lemma
create class EasternYiddish extends Lemma
create class Pitjantjatjara extends Lemma
create class Lojban extends Lemma
create class Luxembourgish extends Lemma
create class Romansch extends Lemma
create class Mingrelian extends Lemma
create class Maori extends Lemma
create class Kara_Kalpak extends Lemma
create class Divehi extends Lemma
create class Assamese extends Lemma
create class Abkhaz extends Lemma
create class Kotava extends Lemma
create class Sundanese extends Lemma
create class Cornish extends Lemma
create class PennsylvaniaDeitsch extends Lemma
create class Comorian extends Lemma
create class Saterfriesisch extends Lemma
create class Luganda extends Lemma
create class Sinhala extends Lemma
create class Bishnupriya extends Lemma
create class CentralKurdish extends Lemma
create class Tigrigna extends Lemma
create class ScottishGaelic extends Lemma
create class OldEnglish extends Lemma
create class Dzongkha extends Lemma
create class LinguaFrancaNova extends Lemma
create class Nhengatu extends Lemma
create class Silesian extends Lemma
create class MeglenoRomanian extends Lemma
create class Ingush extends Lemma
create class SouthernBalochi extends Lemma
create class Erzya extends Lemma
create class Lao extends Lemma
create class Occitan extends Lemma
create class Ligurian extends Lemma
create class Burmese extends Lemma
create class Amharic extends Lemma
create class Moksha extends Lemma
create class Ilokano extends Lemma
create class Chuvash extends Lemma
create class Samoan extends Lemma
create class Udmurt extends Lemma
create class TokPisin extends Lemma
create class Franco_Proven��al extends Lemma
create class Scots extends Lemma
create class Wymysorys extends Lemma
create class Hawaiian extends Lemma
create class Chechen extends Lemma
create class Bosnian extends Lemma
create class Tongan extends Lemma
create class Bakhtiari extends Lemma
create class Pontic extends Lemma
create class EgyptianArabic extends Lemma
create class Picard extends Lemma
create class SouthSaami extends Lemma
create class V��ro extends Lemma
create class K-iche- extends Lemma
create class TabascoChontal extends Lemma
create class Chol extends Lemma
create class Ch-orti- extends Lemma
create class Tzeltal extends Lemma
create class Tzotzil extends Lemma
create class Chicomuceltec extends Lemma
create class Huastec extends Lemma
create class Chuj extends Lemma
create class Tojolabal extends Lemma
create class Jakalteko extends Lemma
create class Kanjobal extends Lemma
create class Akateko extends Lemma
create class Mocho extends Lemma
create class Awakateko extends Lemma
create class Ixil extends Lemma
create class Mam extends Lemma
create class Tektiteko extends Lemma
create class Kekch�� extends Lemma
create class Poqomam extends Lemma
create class Poqomchi- extends Lemma
create class Kaqchikel extends Lemma
create class Achi extends Lemma
create class Tz-utujil extends Lemma
create class Sacapulteco extends Lemma
create class Sipacapense extends Lemma
create class Uspanteco extends Lemma
create class Mop��nMaya extends Lemma
create class Lacandon extends Lemma
create class Yucat��nMaya extends Lemma
create class MiddleHighGerman extends Lemma
create class OldHighGerman extends Lemma
create class Mongolian extends Lemma
create class Aceh extends Lemma
create class RapaNui extends Lemma
create class OldFrench extends Lemma
create class MiddleFrench extends Lemma
create class HaitianCreoleFrench extends Lemma
create class ChurchSlavic extends Lemma
create class Latgalian extends Lemma
create class Aramaic-Official,Hebrew- extends Lemma
create class Aramaic-JewishPalestinian- extends Lemma
create class ClassicalSyriac extends Lemma
create class JewishBabylonianAramaic extends Lemma
create class Hiligaynon extends Lemma
create class OldIrish extends Lemma
create class Karachay_Balkar extends Lemma
create class R��unionCreole extends Lemma
create class Benga extends Lemma
create class Punu extends Lemma
create class Mpongwe extends Lemma
create class K��l�� extends Lemma
create class Chewa extends Lemma
create class Bemba extends Lemma
create class Umbundu extends Lemma
create class Lunda extends Lemma
create class Tonga extends Lemma
create class Albanian extends Lemma
create class Sesotho extends Lemma
create class Kalaallisut extends Lemma
create class NorthernKurdish extends Lemma
create class Kabuverdianu extends Lemma
create class Ugaritic extends Lemma
create class VlaxRomani extends Lemma
create class ClassicalArmenian extends Lemma
create class OttomanTurkish extends Lemma
create class Abi extends Lemma
create class Araki extends Lemma
create class Bislama extends Lemma
create class Fon extends Lemma
create class Futunan extends Lemma
create class International extends Lemma
create class Holikachuk extends Lemma
create class Uzbek extends Lemma
create class Anglo_Norman extends Lemma
create class Bainouk_Gunyaamolo extends Lemma
create class Greek,Ancient-Egypt- extends Lemma
create class Malagasy extends Lemma
create class TsimihetyMalagasy extends Lemma
create class Fala extends Lemma
create class Oriya extends Lemma
create class Lezgian extends Lemma
create class Chamicuro extends Lemma
create class ParaguayanGuaran�� extends Lemma
create class Rohingya extends Lemma
create class TaiDamwritteninLatinscript extends Lemma
create class Tupinamb�� extends Lemma
create class NorthAzerbaijani-Latin- extends Lemma
create class MinNan-POJ- extends Lemma
create class MinNan-simplified- extends Lemma
create class MinNan-traditional- extends Lemma
create class Turkmen extends Lemma
create class OldOccitan extends Lemma
create class Tlingit extends Lemma
create class Hopi extends Lemma
create class Potawatomi extends Lemma
create class Mapudungun extends Lemma
create class AssyrianNeo_Aramaic extends Lemma
create class CentralAymara extends Lemma
create class CentralBicolano extends Lemma
create class Chamorro extends Lemma
create class Filipino extends Lemma
create class Jingpho extends Lemma
create class NorthSaami extends Lemma
create class Jola_Fonyi extends Lemma
create class Swati extends Lemma
create class Tswana extends Lemma
create class Venda extends Lemma
create class Tsonga extends Lemma
create class Squamish extends Lemma
create class Eyak extends Lemma
create class CajunFrench extends Lemma
create class Dhuwal extends Lemma
create class Djinba extends Lemma
create class Koryak extends Lemma
create class Itene extends Lemma
create class Tamasheq extends Lemma
create class Wari- extends Lemma
create class Djamindjung extends Lemma
create class Dido extends Lemma
create class Abaza extends Lemma
create class Wiradhuri extends Lemma
create class MiddleEnglish extends Lemma
create class Tunica extends Lemma
create class AyacuchoQuechua extends Lemma
create class CuscoQuechua extends Lemma
create class MoroccanArabic extends Lemma
create class Omurano extends Lemma
create class Bissau_BolamaCreole extends Lemma
create class ZiguinchorCreole extends Lemma
create class Curassese extends Lemma
create class Aranese extends Lemma
create class X��r��c���� extends Lemma
create class Xaragure extends Lemma
create class Taos extends Lemma
create class Warlpiri extends Lemma
create class StandardXinjiangUyghur extends Lemma
create class Alutor extends Lemma
create class Chukot extends Lemma
create class Yakut extends Lemma
create class Menik extends Lemma
create class Macanese extends Lemma
create class Achumawi extends Lemma
create class Dogrib extends Lemma
create class Gagauz extends Lemma
create class Kalmyk extends Lemma
create class Khakas extends Lemma
create class Komi_Permyak extends Lemma
create class Kumyk extends Lemma
create class Me��nkieli extends Lemma
create class Nanticoke extends Lemma
create class Ngarrindjeri extends Lemma
create class Nottoway_Meherrin extends Lemma
create class Powhatan extends Lemma
create class Tutelo extends Lemma
create class Tuvinian extends Lemma
create class Dolgan extends Lemma
create class Balinese-Latin- extends Lemma
create class Banjar extends Lemma
create class Buginese extends Lemma
create class Madurese extends Lemma
create class MakasarwritteninLatin extends Lemma
create class MakasarwritteninLontara extends Lemma
create class Minangkabau extends Lemma
create class Nias extends Lemma
create class J��rriais extends Lemma
create class Maricopa extends Lemma
create class Nafaanra extends Lemma
create class Nauruan extends Lemma
create class Nogai extends Lemma
create class O-odham extends Lemma
create class Quileute extends Lemma
create class RussiaBuriat extends Lemma
create class Shor extends Lemma
create class SouthernAltai extends Lemma
create class Tahitian extends Lemma
create class Ithkuil-romanized- extends Lemma
create class OldWestNorse extends Lemma
create class OldEastNorse extends Lemma
create class NorthAzerbaijani-Cyrillic- extends Lemma
create class ClassicalNahuatl extends Lemma
create class Tarantino extends Lemma
create class Ainu-Latin- extends Lemma
create class Ainu-Katakana- extends Lemma
create class Abinomn extends Lemma
create class SkoltSami extends Lemma
create class Krio extends Lemma
create class Dalmatian extends Lemma
create class Alabama extends Lemma
create class Tausug extends Lemma
create class LingwadePlaneta extends Lemma
create class MacedoRomanian extends Lemma
create class Mbundu extends Lemma
create class Sercquais extends Lemma
create class MiddleLowGerman extends Lemma
create class Acholi extends Lemma
create class Gikuyu extends Lemma
create class Guern��siais extends Lemma
create class GuerreroAmuzgo extends Lemma
create class SanPedroAmuzgosAmuzgo extends Lemma
create class Archi extends Lemma
create class GuerreroNahuatl extends Lemma
create class Isthmus_MecayapanNahuatl extends Lemma
create class MorelosNahuatl extends Lemma
create class Bor��ro extends Lemma
create class Ute extends Lemma
create class Avestan extends Lemma
create class Eshtehardi extends Lemma
create class Friulian extends Lemma
create class Livonian extends Lemma
create class OldPersian extends Lemma
create class Polabian extends Lemma
create class Veps extends Lemma
create class Aguaruna extends Lemma
create class Tanaina extends Lemma
create class Goguryeo extends Lemma
create class Montagnais extends Lemma
create class SouthSlavey extends Lemma
create class Adyghe extends Lemma
create class OldSaxon extends Lemma
create class MinNan-H��n_l��- extends Lemma
create class Hebrew-nikkud- extends Lemma
create class AkkalaSami extends Lemma
create class Darkinyung extends Lemma
create class Gamilaraay extends Lemma
create class KalaLagawYa extends Lemma
create class Emilian extends Lemma
create class Fijian extends Lemma
create class Judeo_Spanish extends Lemma
create class Rwanda extends Lemma
create class Mazanderani extends Lemma
create class Franc_Comtois extends Lemma
create class Tuvaluan extends Lemma
create class Nama extends Lemma
create class Mizo extends Lemma
create class Lakota extends Lemma
create class Karelian extends Lemma
create class Sasak extends Lemma
create class Shan extends Lemma
create class Iban extends Lemma
create class Ngaju extends Lemma
create class OldJavanese extends Lemma
create class Aleut extends Lemma
create class Pipil extends Lemma
create class CampidaneseSardinian extends Lemma
create class Gallurese extends Lemma
create class Sassarese extends Lemma
create class Tibetan extends Lemma
create class Gothic extends Lemma
create class Micmac extends Lemma
create class Pali-Latinscript- extends Lemma
create class EasternCanadianInuktitut-Latinscript- extends Lemma
create class EasternCanadianInuktitut-Inuktitutsyllabics- extends Lemma
create class Apala�� extends Lemma
create class Baure extends Lemma
create class CentralAtlasTamazight-TifinaghScript- extends Lemma
create class Dakota extends Lemma
create class Ket extends Lemma
create class KildinSami extends Lemma
create class Miyako extends Lemma
create class M��cheno extends Lemma
create class Paumar�� extends Lemma
create class Tabassaran extends Lemma
create class TocharianA-Latinscript- extends Lemma
create class TocharianB-Latinscript- extends Lemma
create class !X���� extends Lemma
create class Chejumal extends Lemma
create class Karata extends Lemma
create class Atikamekw extends Lemma
create class Avar extends Lemma
create class MedievalLatin extends Lemma
create class Main_Franconian extends Lemma
create class Dongxiang extends Lemma
create class Haida extends Lemma
create class NgazidjaComorian extends Lemma
create class SouthernLuri extends Lemma
create class NorthernLuri extends Lemma
create class OldChurchSlavonic-Glagolitic- extends Lemma
create class Malay-Jawi- extends Lemma
create class Seneca extends Lemma
create class Cayuga extends Lemma
create class Mohawk extends Lemma
create class Oneida extends Lemma
create class TorresStraitCreole extends Lemma
create class Swabian extends Lemma
create class Belarusan-Tarakievica- extends Lemma
create class Kabardian extends Lemma
nagarajasr commented 9 years ago

I just ran the script you provided both in 2.1-rc6 release and 2.0.13 release from OrientDB console, the script finished creating all the classes in less than a minute. i'm inclined to think this is an environmental issue. What OS are you using?

iw2rmb commented 9 years ago

Thank you for testing! OS X Yosemite 10.10.4. Don't have any clue what can it be. / If you have any suggestion — i'll check. Otherwise I propose close the issue.

andrii0lomakin commented 9 years ago

Hi, That is bug with your disk + OS performance for forsesynch. If you create all classes before you use DB in your application could you create db in console with settings listed below before using on server. Is that possible ?

Settings are following: -Dstorage.makeFullCheckpointAfterCreate=false -Dstorage.makeFullCheckpointAfterClusterCreate=false -Dstorage.configuration.syncOnUpdate=false

acmeguy commented 9 years ago

Hi,

We have the same problem and determined, after some testing, that is was OS/Storage related and we gave up looking for the reason. I will try the settings from Andrey as well.

Regards, -Stefan

On Mon, 3 Aug 2015 at 07:09 Andrey Lomakin notifications@github.com wrote:

Hi, That is bug with your disk + OS performance for forsesynch. If you create all classes before using could you create db in console with settings listed below and run the same script on console. Before using on server. Is that possible ?

Settings are following: -Dstorage.makeFullCheckpointAfterCreate=false -Dstorage.makeFullCheckpointAfterClusterCreate=false -Dstorage.configuration.syncOnUpdate=false

— Reply to this email directly or view it on GitHub https://github.com/orientechnologies/orientdb/issues/4649#issuecomment-127145107 .

VitalyKovalev commented 9 years ago

Due to logic I can't run test w/o data but seems these options help run faster. Where can I read about them? Do I need to rerun server without them after bulk insertion?

iw2rmb commented 9 years ago

Confirm that with options below and BEFORE creating any indexes script runs less than a minute. Not an issue for me now. Thank you!

In my case there are two indexes on Lemma class. Seems to me index rearrange takes time and last grows with data. May be there is a reason to mention as some rule in docs — create child classes before indexes and data if possible.

Last comment was mine but from old account, sorry.

andrii0lomakin commented 8 years ago

In 2.2 those options are applied automatically so I close issue.