ODB server doesn't respect "storage.wal.path" directive when accessed via plocal

[carloprad]

I added this property to the orientdb-server-config.xml:

but the "WAL" directory remains empty, and the "WAL" files are in the database directory:

root@orientdbZone:~/# ls -lt /opt/orientdb-community-1.7-SNAPSHOT/databases/dbpedia/ total 9187 -rw-r--r-- 1 root root 8978432 Apr 24 15:29 dbpedia.0.wal -rw-r--r-- 1 root root 66560 Apr 24 15:29 nodo.cpm -rw-r--r-- 1 root root 590848 Apr 24 15:29 nodo.pcl [..]

root@orientdbZone:~/# ls -lt /opt/orientdb-community-1.7-SNAPSHOT/wal/ total 0

Here my environment: S.O.: Oracle Solaris 11.1 x86_64 JDK: 1.7.0_51 Oracle 64bit ODB: orientdb-community-1.7-20140424.115531-159-distribution Access type: plocal

[andrii0lomakin]

Hi, Here is result of my testing I had property in orientdb-server-config.xml

I also have created database wal location.

Files in db directory: [andrey@localhost wallocation]$ ls database.ocf e.pcl manindex.pcl orole.cpm OUser.name.sbt default.cpm index.cpm name_id_map.cm ORole.name.sbt ouser.pcl default.pcl index.pcl ofunction.cpm orole.pcl ts.cpm dictionary.sbt internal.cpm ofunction.pcl oschedule.cpm ts.pcl dirty.fl internal.pcl orids.cpm oschedule.pcl v.cpm e.cpm manindex.cpm orids.pcl ouser.cpm v.pcl

and files in wal directory:

[andrey@localhost wal]$ ls wallocation.0.wal wallocation.wmr

Could you:

1. Try using 1.7 release ?
2. Verify that OS user which is used to start OrientDB server has rights to change content of wal directory ?
3. You have created database after property was changed but not before this change.

[carloprad]

Hi Andrey, I just repeated the test using ODB 1.7 community:

• OrientDB server and the loading script are both executed by the root user;
• First I modified the config file;
• Then I started the ODB server in order to create the new database;
• Then I shutdown the ODB
• At last I executed the loading script

Here the details:

• my orientdb-server-config.xml properties section:

    <properties>
        <entry value="1" name="db.pool.min"/>
        <entry value="50" name="db.pool.max"/>
        <entry value="true" name="cache.level1.enabled"/>
        <entry value="false" name="cache.level2.enabled"/>
        <entry value="0" name="cache.level2.size"/>
        <entry value="true" name="profiler.enabled"/>
        <entry value="info" name="log.console.level"/>
        <entry value="fine" name="log.file.level"/>
        <entry value="/opt/orientdb-community-1.7/wal" name="storage.wal.path"/>
    </properties>

• the loading script:

#!/usr/bin/bash
ORIENTDB_HOME="/opt/orientdb-community-1.7"
export ORIENTDB_HOME
/opt/jdk1.7.0_51/bin/amd64/java -cp "$ORIENTDB_HOME/lib/orientdb-server-1.7-SNAPSHOT.jar:$ORIENTDB_HOME/lib/*:." Importer $1 $2 $3 $4

• the Importer source (by Luigi Dell'Aquila):

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;

import com.orientechnologies.orient.core.index.OIndexManagerProxy;
import com.orientechnologies.orient.core.metadata.schema.OClass;
import com.orientechnologies.orient.core.metadata.schema.OSchema;
import com.orientechnologies.orient.core.metadata.schema.OType;
import com.orientechnologies.orient.core.metadata.schema.OClass.INDEX_TYPE;
import com.orientechnologies.orient.core.sql.query.OSQLSynchQuery;
import com.tinkerpop.blueprints.Edge;
import com.tinkerpop.blueprints.Vertex;
import com.tinkerpop.blueprints.impls.orient.OrientGraph;
import com.tinkerpop.blueprints.impls.orient.OrientGraphNoTx;

public class Importer {

        public static void main(String[] args) throws Exception {
                if (args.length != 4) {
                        System.out
                                        .println("sintassi corretta: Importer <nomefile> <dbUrl> <user> <pw>");
                        return;
                }
                creaSchema(args[0], args[1], args[2], args[3]);
                importa(args[0], args[1], args[2], args[3]);
        }

        private static void creaSchema(String fileName, String dbUrl, String user,
                        String pw) throws Exception {
                OrientGraphNoTx db = new OrientGraphNoTx(dbUrl, user, pw);
                OSchema schema = db.getRawGraph().getMetadata().getSchema();
                OIndexManagerProxy indexMgr = db.getRawGraph().getMetadata()
                                .getIndexManager();
                OClass nodo = null;
                if (schema.existsClass("Nodo")) {
                        nodo = schema.getClass("Nodo");
                } else {
                        nodo = schema.createClass("Nodo", schema.getClass("V"));
                }
                if (!nodo.existsProperty("name")) {
                        nodo.createProperty("name", OType.STRING);
                }
                if (!indexMgr.existsIndex("Nodo_name")) {
                        nodo.createIndex("Nodo_name", INDEX_TYPE.UNIQUE, "name");
                }

                OClass arco = null;
                if (schema.existsClass("Arco")) {
                        arco = schema.getClass("Arco");
                } else {
                        arco = schema.createClass("Arco", schema.getClass("E"));
                }
                if (!arco.existsProperty("name")) {
                        arco.createProperty("name", OType.STRING);
                }
                db.shutdown();
        }

        private static void importa(String fileName, String dbUrl, String user,
                        String pw) throws Exception {
                File file = new java.io.File(fileName);
                FileInputStream input = new java.io.FileInputStream(file);
                BufferedReader reader = new java.io.BufferedReader(
                                new java.io.InputStreamReader(input));

                OrientGraph gdb = new OrientGraph(dbUrl, user, pw);

                reader.readLine();
                String result = reader.readLine();
                int i = 0;
                int limit = 10000;
                while (result != null && result.trim().length() > 0) {
                        String[] parts = result.split(" ");
                        if (parts.length > 3 && parts[3].equals(".")) {
                                Vertex vertex1 = createVertex(parts[0], gdb);
                                Vertex vertex2 = createVertex(parts[2], gdb);

                                Edge edge = vertex1.addEdge("Arco", vertex2);
                                edge.setProperty("name", parts[1]);

                                gdb.commit();
//                              if (i > limit) {
//                                      break;
//                              }
                                i++;
                                if (i % 100 == 0) {
                                        System.out.println("importate " + i + " triple");
                                }
                        }
                        result = reader.readLine();
                }
                gdb.commit();
                gdb.shutdown();
        }

        private static Vertex createVertex(String name, OrientGraph gdb) {
                Iterable<Vertex> queryResult = gdb.command(
                                new OSQLSynchQuery<Vertex>("select from Nodo where name = ?"))
                                .execute(name);
                for (Vertex v : queryResult) {
                        return v;
                }
                Vertex result = gdb.addVertex("class:Nodo");
                result.setProperty("name", name);
                return result;
        }
}

• The loading command:

./test.sh /opt/itwiki-20130605-article-categories.txt plocal:/opt/orientdb-community-1.7/databases/dbpedia admin admin

• My WAL directory content:

root@orientdbZone:~/dbpedia# ls -lt /opt/orientdb-community-1.7/wal/
total 0

• My datafiles directory content:

root@orientdbZone:~/dbpedia# ls -lt /opt/orientdb-community-1.7/databases/dbpedia/
total 9769190
-rw-r--r--   1 root     root     134218752 May 28 20:53 Nodo_name3.hib
-rw-r--r--   1 root     root     229639168 May 28 20:53 nodo.pcl
-rw-r--r--   1 root     root     243007488 May 28 20:53 dbpedia.22.wal
-rw-r--r--   1 root     root     256902144 May 28 20:53 arco.pcl
-rw-r--r--   1 root     root     12583936 May 28 20:53 collections_11.sbc
-rw-r--r--   1 root     root     15729664 May 28 20:52 arco.cpm
-rw-r--r--   1 root     root     268500992 May 28 20:52 dbpedia.21.wal
-rw-r--r--   1 root     root           0 May 28 20:52 dbpedia.wmr
-rw-r--r--   1 root     root     10486784 May 28 20:52 nodo.cpm
-rw-r--r--   1 root     root     268500992 May 28 20:51 dbpedia.20.wal
-rw-r--r--   1 root     root     268500992 May 28 20:49 dbpedia.19.wal
-rw-r--r--   1 root     root     268500992 May 28 20:48 dbpedia.18.wal
-rw-r--r--   1 root     root     268500992 May 28 20:47 dbpedia.17.wal
-rw-r--r--   1 root     root     268500992 May 28 20:46 dbpedia.16.wal
-rw-r--r--   1 root     root     268500992 May 28 20:45 dbpedia.15.wal
-rw-r--r--   1 root     root       66560 May 28 20:45 Nodo_name.hit
-rw-r--r--   1 root     root     67109888 May 28 20:45 Nodo_name2.hib
-rw-r--r--   1 root     root     268500992 May 28 20:44 dbpedia.14.wal
-rw-r--r--   1 root     root     268500992 May 28 20:43 dbpedia.13.wal
-rw-r--r--   1 root     root     268500992 May 28 20:42 dbpedia.12.wal
-rw-r--r--   1 root     root     268500992 May 28 20:41 dbpedia.11.wal
-rw-r--r--   1 root     root        1484 May 28 20:40 name_id_map.cm
-rw-r--r--   1 root     root     268500992 May 28 20:39 dbpedia.10.wal
-rw-r--r--   1 root     root     268500992 May 28 20:38 dbpedia.9.wal
-rw-r--r--   1 root     root     268500992 May 28 20:37 dbpedia.8.wal
-rw-r--r--   1 root     root     268500992 May 28 20:36 dbpedia.7.wal
-rw-r--r--   1 root     root     33555456 May 28 20:36 Nodo_name1.hib
-rw-r--r--   1 root     root     16778240 May 28 20:31 Nodo_name0.hib
-rw-r--r--   1 root     root           1 May 28 20:26 dirty.fl
-rw-r--r--   1 root     root        1024 May 28 20:26 OUser.name.nbt
-rw-r--r--   1 root     root       66560 May 28 20:26 OUser.name.sbt
-rw-r--r--   1 root     root        1024 May 28 20:26 dictionary.nbt
-rw-r--r--   1 root     root       66560 May 28 20:26 dictionary.sbt
-rw-r--r--   1 root     root        1024 May 28 20:26 ORole.name.nbt
-rw-r--r--   1 root     root       66560 May 28 20:26 ORole.name.sbt
-rw-r--r--   1 root     root       66560 May 28 20:26 Nodo_name.him
-rw-r--r--   1 root     root        1024 May 28 20:26 e.cpm
-rw-r--r--   1 root     root       66560 May 28 20:26 e.pcl
-rw-r--r--   1 root     root        1024 May 28 20:26 v.cpm
-rw-r--r--   1 root     root       66560 May 28 20:26 v.pcl
-rw-r--r--   1 root     root        1024 May 28 20:26 oschedule.cpm
-rw-r--r--   1 root     root       66560 May 28 20:26 oschedule.pcl
-rw-r--r--   1 root     root        1024 May 28 20:26 ofunction.cpm
-rw-r--r--   1 root     root       66560 May 28 20:26 ofunction.pcl
-rw-r--r--   1 root     root        1024 May 28 20:26 orids.cpm
-rw-r--r--   1 root     root       66560 May 28 20:26 orids.pcl
-rw-r--r--   1 root     root       66560 May 28 20:26 ouser.cpm
-rw-r--r--   1 root     root      132096 May 28 20:26 ouser.pcl
-rw-r--r--   1 root     root       66560 May 28 20:26 orole.cpm
-rw-r--r--   1 root     root      132096 May 28 20:26 orole.pcl
-rw-r--r--   1 root     root        1024 May 28 20:26 default.cpm
-rw-r--r--   1 root     root       66560 May 28 20:26 default.pcl
-rw-r--r--   1 root     root       66560 May 28 20:26 manindex.cpm
-rw-r--r--   1 root     root      132096 May 28 20:26 manindex.pcl
-rw-r--r--   1 root     root       66560 May 28 20:26 index.cpm
-rw-r--r--   1 root     root      132096 May 28 20:26 index.pcl
-rw-r--r--   1 root     root       66560 May 28 20:26 internal.cpm
-rw-r--r--   1 root     root      132096 May 28 20:26 internal.pcl
-rw-r--r--   1 root     root        1608 May 28 20:26 database.ocf
drwxr-xr-x   2 root     root           2 May 28 20:23 luceneIndexes

[andrii0lomakin]

You use embedded database but change setting on server plocal:/opt/ ... those are not related things. If you wish to set properties in embedded database use com.orientechnologies.orient.core.config.OGlobalConfiguration#WAL_LOCATION right in core or use remote: url instead of plocal:

[carloprad]

Can I bypass using a java option like this:

-Dstorage.wal.path="/my/wal/path"

in the server.sh script?

[andrii0lomakin]

Yes.

On Tue, Jun 3, 2014 at 10:28 AM, carloprad notifications@github.com wrote:

Can I bypass using a java option like this:

-Dstorage.wal.path="/my/wal/path"

in the server.sh script?

— Reply to this email directly or view it on GitHub https://github.com/orientechnologies/orientdb/issues/2285#issuecomment-44928050 .

Best regards, Andrey Lomakin.

Orient Technologies the Company behind OrientDB

[carloprad]

I've just modified the $ORIENTDB_HOME/bin/server.sh file adding the following lines, but it still ignores the directive:

WAL_PATH=$ORIENTDB_HOME/wal
$JAVA $JAVA_OPTS $JAVA_OPTS_SCRIPT $ORIENTDB_SETTINGS -Dstorage.wal.path="$WAL_PATH" -Djava.util.logging.config.file="$LOG_FILE" -Dorientdb.config.file="$CONFIG_FILE" -Dorientdb.www.path="$WWW_PATH" -Dorientdb.build.number="UNKNOWN@r${buildNumber}; 2014-05-27 01:39:25+0200" -cp "$ORIENTDB_HOME/lib/orientdb-server-1.7.jar:$ORIENTDB_HOME/lib/*" com.orientechnologies.orient.server.OServerMain

[andrii0lomakin]

you do not use server in your loading. in script you provide url "plocal:/opt/orientdb-community-1.7/databases/dbpedia" which means embedded not server database, could you either provide url with remote: prefix instead of plocal: or set system property in test.sh script ?

[carloprad]

Solved! I have to specify the property in test.sh script:

ORIENTDB_HOME="/opt/orientdb-community-1.7"
WAL_PATH=$ORIENTDB_HOME/wal
export ORIENTDB_HOME WAL_PATH

/opt/jdk1.7.0_51/bin/amd64/java -Dstorage.wal.path="$WAL_PATH" -cp "$ORIENTDB_HOME/lib/orientdb-server-1.7-SNAPSHOT.jar:$ORIENTDB_HOME/lib/*:." test/Importer $1 $2 $3 $4

Thanks Andrey!