Url /datasets/manager does not exist

[FabricioRojas]

We have several mistakes like this, first of all we found that the user needs to have access to this page, so we tried to set the user as the dataset manager, but there weren't any group or role that fit this request. Anyway we created the group "Database Manager" but we couldn't assign it any permission, the list were empty.

After that we conclude that there are missing some important and required data that are required to get the app working. Or maybe we are just missing some required process before creating the roles.

[ocrasborn]

Correct me if I'm wrong, @Woseseltops / @vanlummelhuizen , but access rights to the dataset are also specified at the level of that dataset; just adding a user to this group does not do the trick, if you don't also specify write access for that user for the dataset. This can be done in the Django admin (button at top right of a dataset's page, forgot the name).

[susanodd]

The user needs to be in the group Dataset_Manager (with an underscore) (not Database Manager).

[susanodd]

I tried what I just suggested on my machine. (To see if the group not existing can cause the bug you show.) But the code prints a message that the group Dataset_Manager isn't found. (It fails gracefully and reports this to the user.)

Can you put some print statements in the get_queryset method of class DatasetManager in signbank/dictionary/adminviews.py. See if you can get down to the line with ObjectPermissionChecker. (See if that line is causing the crash.) You need to have the library django-guardian installed (in your virtual environment). Django also creates database tables for all the permissions.

[You can browse your database using e.g. Sqliteman, to see what tables are in the database. You will see a bunch of Django specific tables such as auth_group, auth_user, auth_permission, etc. These are the tables used by Django guardian. You can check for yourself that you have these tables and that there is data in them. Make sure you close the database browser again.]

To help us help you, can you see if that line of code (ObjectPermissionChecker) is working? If guardian isn't installed it won't find the tables.

[FabricioRojas]

After trying all points suggested by @susanodd, I'm back with some new information. First of all this is absolutely true, but as we didn't get any message we couldn't figure out before (Thanks @susanodd):

The user needs to be in the group Dataset_Manager (with an underscore) (not Database Manager).

We tried to put the prints but we didn't get any message back, so maybe that is the problem.

We checked if django-guardian was installed on the system and so it was, after that we checked if the tables you mentioned above were on the database and also they were. There is something else we are missing?

Finally we also are getting a similar error in signs/show_all/ maybe the reason is related to the other one.

[susanodd]

@FabricioRojas can you do a "pip freeze" inside of your virtual environment and compare that to the required python packages? Just to make sure everything is installed for python.

Another thing to check is the Ajax calls. You can find these in the dictionary/urls.py file. Try some of these in the browser to confirm whether they are generating output. Many of the pages rely on calls to the database via Ajax so it's more efficient.

A fun one to try: (you need to be logged in for this example) https://signbank.science.ru.nl/dictionary/ajax/gloss/ge

(where "ge" - without any / afterwards! - is a pattern for initial letters of glosses.

This ajax call is used in templates where the user needs to choose a gloss. The ajax call returns json for the matches of glosses that start with "ge". You can adapt this to your own data. Do such an example to confirm that 1. ajax is running, and 2. you actually have some data. (Choose a couple of letters where you know there are signs for in your database.)

[logic-byte]

@susanodd After compare pip freeze with the requirements, everything looks fine. The ajax call seems to work too. But in some cases, like when generate_translated_choice_list_table fails to get data in /signbank/dictionary/templates/dictionary/admin_gloss_list.html, the TypeError appears. It's like we are missing some content on the DB.

Is this normal? Should we create everything that is missing by hand? Or there is maybe some DB that is used as template in order to don't get this errors?

[susanodd]

How many languages are you working with? (Just to check if there are more than one, or just one. The code normally assumes that English is always there. In some places, English is used as the default. You can compare the signbank/settings/server_specific for asl_yale.py to those for global_sb_new_aj.py) ASL only uses one language. There are some settings that also have mappings for translation languages. Take a look at the settings for LANGUAGES, LANGUAGE_CODE, DEFAULT_KEYWORDS_LANGUAGE, DEFAULT_LANGUAGE_HEADER_COLUMN, and LANGUAGE_CODE_MAP.. Notice how English appears. Then compare to the ASL settings where there is only one language. See if your settings are totally different. Report if you find anything you need differently. These settings are used for generating choices.)

How many datasets do you have? In the settings, does the DEFAULT_DATASET_LANGUAGE_ID match up with the correct ID? (You need to see what it is in your database.)

I'm working on the admin_gloss_list page right now. A new version is nearly ready. I noticed some other things that were (are broken online) like the number of pages to display. Can you roughly say what kind of query you were doing when you got the error? Had you filled in a Search query?

The generated choice list table is for the context variables. Because it takes a lot of time to display things, we pre-fetched some of the data to make the display faster. It sounds like someplace it's getting an empty list for something. There could be a None value returned someplace that it's expected to have a value. Can you see anything else useful in the error display? Like what line of code was executed before it got the TypeError? Was it during generation of the context or during the query? (You could put some print statements in to see how far it got before crashing.)

There was something messed up with the page numbers. It's not displaying the selection correctly because there was (is) a string (from the template) instead of an int (for the python code). It looks like some of the previous versions of python gracefully typecast things, which have become errors. Because the software has been developed for more than 10 years, there are places with older code and python is also changing.

It could be that something was previously typecast but now gets an error. (The get from the request code that's online now is broken for paginate_by. It used to work. The new version - not yet pushed - has a conditional instead of a one-line get. That's where the type casting from string to int wasn't working anymore.)

But it's more likely that something isn't set properly in the settings.

Any additional information about the error you can provide will help.

[susanodd]

The changes to admin_gloss_list I mentioned above have been pushed to github. (They're not on the global live server yet.)

[Delianpy]

Hi, I'll answer some questions that @susanodd asked.

How many languages are you working with? (Just to check if there are more than one, or just one. The code normally assumes that English is always there. In some places, English is used as the default.

In the database there are three languages: Catalan (ID=1), Spanish (ID=2) and English (ID=3), and in any datasets or the file global_sb_new_aj.py you can see that English is never used.

In the file /signbank/settings/server_specific/global_sb_new_aj.py, DEFAULT_DATASET_LANGUAGE_ID is set on 1 (Catalan)

The default language (and the only one that exists) on the file /signbank/settings/server_specific/asl_yale.py is English, the only language that ASL uses for the project.

How many datasets do you have? In the settings, does the DEFAULT_DATASET_LANGUAGE_ID match up with the correct ID? (You need to see what it is in your database.)

We can see that there are four datasets on the table dictionary_dataset.

[Delianpy]

/signs/show_all/ displayed errors because the view tries to access a value called handedness that didn't existes in the dictionary generate_translated_choice_list_table. When added the Field Choice called Handedness in de admin panel the error is solved, but now there are other similar problem.

The value to have problems now is called locprim (The value that should be displayed in the column Location in the table of the view). We tried to do the same thing to solve it, but doesn't work. Also, we tried to put some value in the column locprim of the table dictionary_gloss of the database, but the error still persist.

Is there any way to add the value to where it belongs? Or are we missing something?

[susanodd]

Which version of Signbank did you originally install? Global? The Handedness field is in the Gloss model. It should have been put in the database tables in the correct place (automatically). You are mentioning Field Choices. Where did your original content come from?

Did you migrate your database after changing models if you made changes? Also, the server needs to be restarted after adding new Field Choices. Were your Field Choices originally empty, or did you get them from somewhere? @vanlummelhuizen created an initial database for you when you first started. (#562). Did you migrate that database after updating models? (You need to generate migration files and apply them to the database. Django isn't that smart.) That could be an issue that your database needs to be migrated to correspond with changes. Or it could be that Field Choices were not in your original database?

[Delianpy]

We migrate the database after changing the models and yes, effectively, the Field Choices "Handedness" and "Locprim" were not in the database @susanodd, they weren't been put in the database automatically.

[Delianpy]

After reading carefully the code of the function generate_translated_choice_list_table() in /signbank/dictionary/models.py, we realized that "locprim" is the key of the array that returns that function, while the field choice must be called Location. Added the field choice with that name and is already shown in the complete table in /signs/show_all/.

One important point is that we had to add the field choice handedness and location manually.

[susanodd]

@Delianpy, thank for identifying the problem. It seems that your initial database did not have any, or insufficient data in FieldChoice. Because Signbank has evolved, it did not occur before that this table might not have basic data in it. We had issues when Handshapes were added. There is code for creating the initial data if it's not there. It would seem something similar is needed if FieldChoice has no data. We need to add some feedback when such an error arises, instead of crashing (as you experienced).

When you first set up Signbank, which of the settings files did you use? You mention the languages, but you also mention the Yale settings file. Since you're multi-lingual you should not be using the Yale one.

Check these settings (here, from Global) (make sure the "English" part is changed to Catalan. The language codes are from Django. The 'name_en' field is from FieldChoice. Some of the Analysis routines rely on English for display. If anything in these settings is causing problems, let us know. It would mean something needs to be more multi-lingual):

`LANGUAGES = ( ('en', gettext('English')), ('nl', gettext('Dutch')), ('zh-hans', gettext('Chinese')) ) LANGUAGE_CODE = "en"

SEPARATE_ENGLISH_IDGLOSS_FIELD = True

DEFAULT_KEYWORDS_LANGUAGE = {'language_code_2char': 'en'}

DEFAULT_LANGUAGE_HEADER_COLUMN = {'English': 'name_en'}`

[susanodd]

@FabricioRojas I was able to reproduce your TypeError NoneType error. It's exactly as @Delianpy described above. If there are no field choices for Handedness, then no data is put in the table. I added an else clause to make sure the first two choices ('-', and 'N/A') are put in the table. Indeed, if no Field Choices were found, these choices were not put in the table. Leading to a 'NoneType' error.

[FabricioRojas]

Thank you @susanodd , we'll merge that locally in order to have the updated code.

What we have until now is a several time patched project that it could work in the tested cases, but maybe it couldn't on others. We also continiue with an un-updated DB which has several missing changes

How should we continue now? Should we start from 0 all the process in order to get a completely functional project? Or we just need an updated DB?

We finally hope we could get the full project working properly soon.

[susanodd]

I would do something with your database to construct an initial state that contains all the necessary "starting point" data. Then start editing and adding glosses to that state.

The NGT signbank has evolved continuously. There are actually some initial migrations that contain things that have become obsolete. Django likes to apply a sequence of migrations to things. This was causing problems with testing, because it kept wanting to start with the initial database and apply migrations to it before beginning with the tests. @Woseseltops discovered that there was a huge amount of data in the test database because Django was doing that. We now do the tests on a minimal database and create glosses, etc. we want to test. We always make a new copy of the most recent database, then apply the create_test_db to that copy, so the tests always begin with an initial state.

You can take a look at the create_test_db.py file (under dictionary/management/commands) to see what tables (data) are "kept". Although that's not directly visible, only the data that are deleted are in the code. Those are objects that depend on each other with relations. (So far this isn't documented very well.)

You could look at your own database and leave the structure, export the glosses you have created so far, then clean up the database and make an initial state that contains all the necessary things. Then let Django start migrations from that. And create glosses in your system from that. @Woseseltops knows more about this.

You always need an initial state. Then Django will create the needed migrations. But make sure you don't have unnecessary migrations in case it's been doing weird things when things were missing. Our migrations we always trim them down to only the things that have been updated to the model. Django likes to add lots of extra stuff to the migrations. You could take a look at our more recent migrations to see what minimal stuff is there. Then look at some of the initial migrations and see that there is tons of stuff.

This is a bit chatty. You can always make copies of things in a place that Django doesn't look. To keep a record of how your system was built. If you delete migrations, then generate migrations again from the current code. Django will migrate between your initial migration and your code. But you need your database to match the migration you start from. It's easier if you can get an initial database and an initial migration starting point before your actual data starts being stored, to avoid Django wanting to migrate your actual data from databases that weren't working yet.

Hopefully that makes sense.

[FabricioRojas]

Thank you for the response.

If I we understood it correctly, we should backup our DB in order to clean it up and then leave the main structure that Django needs for the migration process. After all this procees we'll get a fully funcional DB isn't it?

If I'm not wrong we tried this method at first but it failed, then we wrote this issue and thus was we get the current database that we have.

There is any chance that you could give to us a functional schema so we could skip the migration process?

[Woseseltops]

If I we understood it correctly, we should backup our DB in order to clean it up and then leave the main structure that Django needs for the migration process. After all this procees we'll get a fully funcional DB isn't it?

Hi @FabricioRojas, not sure if I fully understand what you mean, so let me explain this to make sure we are all on the same page:

• If you change a Django model (for example, add a field to the gloss model), the underlying database structure does not match this model anymore. Migrating means changing an exisiting non-empty database to match the Django model again.
• The initial migration is when Django analyzes all defined models and creates a matching database for it. This process currently does not work for Signbank, because during migration Django already requires things to be present in the database for some strange reason (see #564).
• An unrelated problem we had with the Global Signbank is that it contained so much data that the unit tests were super slow. To fix this, we run our tests on a copy of the database where a lot of irrelevant data is removed.

There is any chance that you could give to us a functional schema so we could skip the migration process?

Given the second bullet, I guess what you want is not only a functional schema, but an almost-empty database with the necessary tables already filled? Or even better, I could pick up #564 so the initial migration DOES work? If you want I can have it finished by August 9 (given that I don't run into a lot of unexpected problems).

[FabricioRojas]

Hi @Woseseltops , we were talking about the same.

What we want is an empty database just with the necessary data to run the app without errors. Later we'll fill it with the data.

We could wait until August 9 in order to get the initial migration working, but if you already have an DB schema that we could just import it and get the app working could be an quicker option.

[Woseseltops]

Okay clear @FabricioRojas . Unfortunately, I probably also won't be able to get that database ready before August 9, but I'll try.

[Woseseltops]

Hmm looks like I made a mistake checking my calendar, and I'm not working August 9. Sorry, everywhere I said 9 it should be 14 - so even later :( .

[Woseseltops]

Okay @FabricioRojas , I did #564 . If you now run develop.py migrate a fresh database is created that works out of the box. It has some default things (language, sign language, dataset, pages), but all can be edited . I've attached an example of such a fresh database. Let us know if that solves your problem.

signbank.db.zip

[marcelobtz]

@Woseseltops We tested the database you sent us, we performed the migration and everything seemed to be working fine, but when creating a test page the following error is generated.

Internal Server Error: /signs/show_all/ Traceback (most recent call last): File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response response = self._get_response(request) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 217, in _get_response response = self.process_exception_by_middleware(e, request) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 215, in _get_response response = response.render() File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/response.py", line 107, in render self.content = self.rendered_content File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/response.py", line 84, in rendered_content content = template.render(context, self._request) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/backends/django.py", line 66, in render return self.template.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 207, in render return self._render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/test/utils.py", line 107, in instrumented_test_render return self.nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/loader_tags.py", line 177, in render return compiled_parent._render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/test/utils.py", line 107, in instrumented_test_render return self.nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/loader_tags.py", line 72, in render result = block.nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 322, in render return nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 322, in render return nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 216, in render nodelist.append(node.render_annotated(context)) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 529, in render return self.nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 529, in render return self.nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 529, in render return self.nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 529, in render return self.nodelist.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 990, in render bit = node.render_annotated(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 957, in render_annotated return self.render(context) File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 527, in render six.iteritems(self.extra_context)} File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/defaulttags.py", line 526, in <dictcomp> values = {key: val.resolve(context) for key, val in File "/var/www/global-signbank-test/venv/lib/python3.5/site-packages/django/template/base.py", line 736, in resolve new_obj = func(obj, *arg_vals) File "/var/www/global-signbank-test/signbank/dictionary/templatetags/annotation_idgloss_translation.py", line 84, in get_item return dictionary.get(key) AttributeError: 'str' object has no attribute 'get'

[Woseseltops]

Hi @marcelobtz , can you give us a bit more info on what exactly you were trying to do? Your screenshot and error message are a bit confusing to me:

• In the URL bar of the screenshot I see 'signs/show_all/', where I would expect a list view (overview of signs).
• The error message is about the template tag 'get_item', which is only used in the detail view (show all info on one particular sign).
• To be more specific, the problem seems to be that the 'translated_choice_list', which is a dictionary, is not there, but is a string instead.

About this translated_choice_list: each sign, which is called 'gloss' in the code, has multiple fields, like 'handedness'. The possible values for such a field can be defined in the admin and are called FieldChoices. In the various interface languages, these FieldChoices have different names; the translated_choice_list is a giant dictionary with all FieldChoices translated to all interface languages. Your error occurs when trying to get correct translation from this dictionary.

[susanodd]

@marcelobtz, can you identify some pages that do work okay from those that don't?

Does GlossDetailView work okay? Can you view a single gloss?

Does the Signs -> Search work?

The Signs -> Search works just like the show_all, except it hasn't done any query yet before it's viewed. Make sure it works okay too. Internally, the show_all shares the same code as the Search.

You can put a few print statements at the start of the various get_context_data and get_queryset methods on pages (views) that are giving errors to identify whether the problem is on the template side, or on the python side that's supposed to feed the data to the template.

Sometimes simply a misplaced character can cause obscure errors.

As an example, if one of the javascript constants hasn't got filled with any data, it just has an empty value. Sometimes no errors are reported by the browser and it just ignores the rest of the code. You can either put print statements inside of the get_context_data to make sure data has been filled in, and/or in the javascript to make sure it's been filled in there too. At the very least a print or console.log at the start and end to confirm that it's done the entire block.

[susanodd]

@marcelobtz, could you try removing old compiled pyc binary files? I tried doing a search on various annotation_idgloss_translation things, like the filters registered in that file. My computer finds a match to get_item in an old pyc binary file. The line number of your screenshot doesn't actually correspond to the get_item filter it's complaining about in the current version of that file. I have no idea if it's possible for python to link itself to old compiled code. Since you have reinstalled everything, perhaps double check that there aren't any obsolete binary files. Just in case something weird like that is possible.

[marcelobtz]

Hi @Woseseltops. I just create a simple page in admin with the path "signs/show_all/" asigned to "Signs" menu, nothing more, but this should generate an error?? considering that it is a simple page, or "Signs" menu have a special configuration or something like that??. I ask this question because for example I created a static page in "about/manual", "About" is the main menu and "Manual" is the submenu that contain a static page and the page it shows correctly.

We cannot go to the "signs/show_all/" page for some reason. I suppose that is a database problem.

Hi @susanodd. I can't verify the pages you are saying because the database that @Woseseltops attached to us is a basic database and just contain this page "Signs -> Add new gloss" and empty home page. We already remove old pyc binary files but we still have the same error.

The first thing we want is to be able to view the page "signs/show_all/". I will attach the main page so you can see what we really have.

[susanodd]

@marcelobtz Gee, that's scary! It really is empty.

I can see in the signbank.db file that there is indeed only that page. Did you export the database you were working with in the beginning? Can you open that old database and export SQL statements for the data? INSERT INTO .... VALUES .... Then put back your pages data (the navigation menus). (Either just typing it over in the admin or using SQL.)

Did it work to add a new gloss? The one page that's in the database, does that work?

I'm working on a unit test to check whether necessary FieldChoice data is in the database. I noticed that the empty signbank.db.zip file has no FieldChoice data in it either. If it's not working to create a new gloss, that is probably why. From your screenshot, something to test for pages is also needed.

[susanodd]

I suppose I could try the same as @Woseseltops did and start with the empty database he made and start filling it tables with minimal information so much as is needed to get the unit tests to pass. They are built on a minimal database. Our test-database.db has pages in it, of course.

[susanodd]

@marcelobtz, another thing to check, the signbank.db file should be group and owner e.g., www-data on Ubuntu rather than a specific user. If it's set to e.g., yourself, the server can't open it.

[susanodd]

@marcelobtz, I'm looking at sql statement exports of the schema and data from our test database, as well as from the initial database. Are you planning to develop your own set of pages? Or do you want to start from Signbank Global? The same for FieldChoice. I can make an sql file with INSERT INTO ... VALUES ... statements, although there are quite a lot of them. What interface languages are you using? At the moment, we have Dutch, English, Chinese as interface languages, so there are entries for these in the FieldChoice and Pages tables.

[susanodd]

With regards to getting the page to display, your display isn't showing datasets in the banner across the top. Another source of data that gets displayed is from the headers. You can find these under signbank/templates/global-templates baselayout.html menu.html

These files assume data from the context. viewable_datasets If you look at the method: signbank/tools.py -> get_selected_datasets_for_user This needs to fill in the required data for the banner.

So you can see you need at least one user who has view_dataset permission for one dataset. You can put a couple of print statements in this method to confirm there is data. Then this dataet acronym ought to be showing up in the banner when the user is logged in.

[marcelobtz]

@susanodd I did some tests and I will explain the process I did as detailed as posible.

First, I created a dataset and assign a owner and then I could see the dataset in the header.

I created a new sign via "Signs -> Add new gloss", but in a "ID Gloss" field I created new because I can't create new gloss via admin. Then I see a page that suppose would be a sign page.

But I still have the same error in "Signs -> Show all signs" page. I think some data is missing, for example in admin in Field Choices I don't have any data, or simply some data is missing in the database.

You can see the dataset in the header.

Page of the sign created.

Error when I want create a new gloss in the admin.

Dictionary section with the records of each subsection.

We need clean installation of Global SignBank

[susanodd]

@marcelobtz, thanks for all the screenshots! I'm constructing an sql script with insert commands for the FieldChoice and Pages tables.

Do you already have proper values in the dictionary_language, dictionary_dataset, dictionary_signlanguage, dictionary_dataset_translation_languages ? You did these via the admin or by hand?

I'm using the empty database @Woseseltops created and adding things to it to get sufficient data in it to run the tests. I'm comparing the test database to the empty one to double check what needs data in it. Then just those inserts will be in an sql file to apply to your (initial) database.

I noticed some fields for Pages are specified for Chinese in the models. I'm removing Chinese data from the Pages inserts. Ideally you'd probably want different field names to match your languages. But... So far, we have not done that. It would be a migration for Django, because Django needs to match your database, and the empty database you have now matches the Django/Python code. So it's not possible to start renaming fields in the SQL. You'd need to create a migration to do that. (And delete most of the content from the automatically generated migration file, except for specific commands to rename those fields.)

First I'll get the extra SQL inserts made.

Probably we will need to write a script that lets a new Signbank setup specify what the interface languages are, what the default dataset is, and what the default sign language is, and what the translation languages are for that sign language/dataset.

[susanodd]

@marcelobtz, your screenshot of the error from GlossAdmin, complaining about dataset...

This is caused by a more recent evolution of Signbank. The dataset field is accessed indirectly via the Lemma that the gloss refers to.

Before creating a new Gloss, you will need to create a new Lemma for the Dataset. Then your new Gloss will refer to that Lemma and obtain the dataset from there.

If you look at dictionary/tests.py -> test_CRUD you will see that first, a new Lemma is created. the new Gloss then refers to that Lemma.

[susanodd]

@marcelobtz, I've made an SQL file to put some essential things in the initial empty database. I started with the database @Woseseltops created. Here's the SQL file.

essential-creation-signbank.sql.gz

With that file run sqlite3 on the command line with your initial database:

sqlite3 signbank-initial.db sqlite> .read essential-creation-signbank.sql

Run the SQL using ".read" (with that dot before read). That will insert an amount of data into various database tables. For the FieldChoice table, I've put circa 4 entries into the tables. You'll need to use admin to adjust these to your own data. Some essential parts

1. FieldChoice FingerSelection is a constant table. This is abbreviations for the TIMRP data, which are bits in the representation. This is needed for display.
2. Some Handshapes are in FieldChoice as well as the Handshape table.
3. pages_page has the essential navigation, but the display html that was in our database has been removed, it's just '' fields now. The relevant part is the path, which directs Django
4. auth_group has some entries for groups. I noticed in running the tests that the group Dataset_Manager is needed for the tests

I'd try running this on the database @Woseseltops created first. Then you can compare that to whatever languages and datasets you already created. In making this, some revisions were needed to the "id" in the first column because I kept encountering duplicate ids. So I revised it so that it works with the initial database. And it works with e.g., the Handshapes in the tests. In order to get the unit tests working, the following settings had to be used, to correspond to the initial database.

DEFAULT_DATASET = 'Test Dataset'
DEFAULT_DATASET_ACRONYM = 'TestDS'
DEFAULT_DATASET_LANGUAGE_ID = 1
DEFAULT_DATASET_PK = 4

---

Those match the data in the inital database plus the essential sql data. Feel free to modify those to match your own setup once it's working.

Only one test did not work. That for moving video files. The test creates a video and needs to delete it again from the real system. That one isn't removing the temp file yet. @vanlummelhuizen recently revised the video storage system and it has something to do with that.

Let me know if this works. I will try to come up with something for the discrepancy of having Dutch and Chinese fields. But we probably need to do that with @Woseseltops.

[marcelobtz]

Hi @susanodd. I followed the steps you mentioned and now all views are working fine. But we noticed that there is a new section in the admin in the dictionary section called "Gloss revisions". That view generated the error of the image below. That table doesn't exist in the database.

Would there be some way to pass us the .sql of the structure of that table to add it to ours?.

Thanks for the help.

[susanodd]

@marcelobtz, can you apply the migrations to your database? The code continues to be developed, so when you pull new versions, you also need to apply any corresponding migrations. The GlossRevision was added in the migration from 30 august. You can see migrations in signbank/dictionary/migrations