Error: IndexSizeError: DOM Exception 1 Error: Index or size was negative, or greater than the allowed value.

TMWong commented 11 years ago

I got this error message when I try the embedded visualization feature:

The error can be reproduced using the live embedding page at http://brat.nlplab.org/embed.html, with {"entity_types": [{ "type": "test", "labels": [ "test", "test" ], "bgColor": "#7fa2ff", "borderColor": "darken" }]}

and

{ "text": "(Inaugural Address By President Barack Hussein Obama: http://www.whitehouse.gov/blog/inaugural-address/)\n\nMy fellow citizens: I stand here today humbled by the task before us, grateful for the trust you''ve bestowed, mindful of the sacrifices borne by our ancestors. \nI thank President Bush for his service to our nation -- (applause) -- as well as the generosity and cooperation he has shown throughout this transition.\nForty-four Americans have now taken the presidential oath. The words have been spoken during rising tides of prosperity and the still waters of peace. Yet, every so often, the oath is taken amidst gathering clouds and raging storms. At these moments, America has carried on not simply because of the skill or vision of those in high office, but because we, the people, have remained faithful to the ideals of our forebears and true to our founding documents. \nSo it has been; so it must be with this generation of Americans.\nThat we are in the midst of crisis is now well understood. Our nation is at war against a far-reaching network of violence and hatred. Our economy is badly weakened, a consequence of greed and irresponsibility on the part of some, but also our collective failure to make hard choices and prepare the nation for a new age. Homes have been lost, jobs shed, businesses shuttered. Our health care is too costly, our schools fail too many -- and each day brings further evidence that the ways we use energy strengthen our adversaries and threaten our planet.\nThese are the indicators of crisis, subject to data and statistics. Less measurable, but no less profound, is a sapping of confidence across our land; a nagging fear that America''s decline is inevitable, that the next generation must lower its sights.\nToday I say to you that the challenges we face are real. They are serious and they are many. They will not be met easily or in a short span of time. But know this America: They will be met. (Applause.)\nOn this day, we gather because we have chosen hope over fear, unity of purpose over conflict and discord. On this day, we come to proclaim an end to the petty grievances and false promises, the recriminations and worn-out dogmas that for far too long have strangled our politics. We remain a young nation. But in the words of Scripture, the time has come to set aside childish things. The time has come to reaffirm our enduring spirit; to choose our better history; to carry forward that precious gift, that noble idea passed on from generation to generation: the God-given promise that all are equal, all are free, and all deserve a chance to pursue their full measure of happiness. (Applause.)\nIn reaffirming the greatness of our nation we understand that greatness is never a given. It must be earned. Our journey has never been one of short-cuts or settling for less. It has not been the path for the faint-hearted, for those that prefer leisure over work, or seek only the pleasures of riches and fame. Rather, it has been the risk-takers, the doers, the makers of things -- some celebrated, but more often men and women obscure in their labor -- who have carried us up the long rugged path towards prosperity and freedom. \nFor us, they packed up their few worldly possessions and traveled across oceans in search of a new life. For us, they toiled in sweatshops, and settled the West, endured the lash of the whip, and plowed the hard earth. For us, they fought and died in places like Concord and Gettysburg, Normandy and Khe Sahn. \nTime and again these men and women struggled and sacrificed and worked till their hands were raw so that we might live a better life. They saw America as bigger than the sum of our individual ambitions, greater than all the differences of birth or wealth or faction.\nThis is the journey we continue today. We remain the most prosperous, powerful nation on Earth. Our workers are no less productive than when this crisis began. Our minds are no less inventive, our goods and services no less needed than they were last week, or last month, or last year. Our capacity remains undiminished. But our time of standing pat, of protecting narrow interests and putting off unpleasant decisions -- that time has surely passed. Starting today, we must pick ourselves up, dust ourselves off, and begin again the work of remaking America. (Applause.)\nFor everywhere we look, there is work to be done. The state of our economy calls for action, bold and swift. And we will act, not only to create new jobs, but to lay a new foundation for growth. We will build the roads and bridges, the electric grids and digital lines that feed our commerce and bind us together. We''ll restore science to its rightful place, and wield technology''s wonders to raise health care''s quality and lower its cost. We will harness the sun and the winds and the soil to fuel our cars and run our factories. And we will transform our schools and colleges and universities to meet the demands of a new age. All this we can do. All this we will do.\nNow, there are some who question the scale of our ambitions, who suggest that our system cannot tolerate too many big plans. Their memories are short, for they have forgotten what this country has already done, what free men and women can achieve when imagination is joined to common purpose, and necessity to courage. What the cynics fail to understand is that the ground has shifted beneath them, that the stale political arguments that have consumed us for so long no longer apply. \nThe question we ask today is not whether our government is too big or too small, but whether it works -- whether it helps families find jobs at a decent wage, care they can afford, a retirement that is dignified. Where the answer is yes, we intend to move forward. Where the answer is no, programs will end. And those of us who manage the public''s dollars will be held to account, to spend wisely, reform bad habits, and do our business in the light of day, because only then can we restore the vital trust between a people and their government.\nNor is the question before us whether the market is a force for good or ill. Its power to generate wealth and expand freedom is unmatched. But this crisis has reminded us that without a watchful eye, the market can spin out of control. The nation cannot prosper long when it favors only the prosperous. The success of our economy has always depended not just on the size of our gross domestic product, but on the reach of our prosperity, on the ability to extend opportunity to every willing heart -- not out of charity, but because it is the surest route to our common good. (Applause.)\nAs for our common defense, we reject as false the choice between our safety and our ideals. Our Founding Fathers -- (applause) -- our Founding Fathers, faced with perils that we can scarcely imagine, drafted a charter to assure the rule of law and the rights of man -- a charter expanded by the blood of generations. Those ideals still light the world, and we will not give them up for expedience sake. (Applause.)\nAnd so, to all the other peoples and governments who are watching today, from the grandest capitals to the small village where my father was born, know that America is a friend of each nation, and every man, woman and child who seeks a future of peace and dignity. And we are ready to lead once more. (Applause.)\nRecall that earlier generations faced down fascism and communism not just with missiles and tanks, but with the sturdy alliances and enduring convictions. They understood that our power alone cannot protect us, nor does it entitle us to do as we please. Instead they knew that our power grows through its prudent use; our security emanates from the justness of our cause, the force of our example, the tempering qualities of humility and restraint.\nWe are the keepers of this legacy. Guided by these principles once more we can meet those new threats that demand even greater effort, even greater cooperation and understanding between nations. We will begin to responsibly leave Iraq to its people and forge a hard-earned peace in Afghanistan. With old friends and former foes, we''ll work tirelessly to lessen the nuclear threat, and roll back the specter of a warming planet.\nWe will not apologize for our way of life, nor will we waver in its defense. And for those who seek to advance their aims by inducing terror and slaughtering innocents, we say to you now that our spirit is stronger and cannot be broken -- you cannot outlast us, and we will defeat you. (Applause.)\nFor we know that our patchwork heritage is a strength, not a weakness. We are a nation of Christians and Muslims, Jews and Hindus, and non-believers. We are shaped by every language and culture, drawn from every end of this Earth; and because we have tasted the bitter swill of civil war and segregation, and emerged from that dark chapter stronger and more united, we cannot help but believe that the old hatreds shall someday pass; that the lines of tribe shall soon dissolve; that as the world grows smaller, our common humanity shall reveal itself; and that America must play its role in ushering in a new era of peace.\nTo the Muslim world, we seek a new way forward, based on mutual interest and mutual respect. To those leaders around the globe who seek to sow conflict, or blame their society''s ills on the West, know that your people will judge you on what you can build, not what you destroy. (Applause.) \nTo those who cling to power through corruption and deceit and the silencing of dissent, know that you are on the wrong side of history, but that we will extend a hand if you are willing to unclench your fist. (Applause.)\nTo the people of poor nations, we pledge to work alongside you to make your farms flourish and let clean waters flow; to nourish starved bodies and feed hungry minds. And to those nations like ours that enjoy relative plenty, we say we can no longer afford indifference to the suffering outside our borders, nor can we consume the world''s resources without regard to effect. For the world has changed, and we must change with it.\nAs we consider the role that unfolds before us, we remember with humble gratitude those brave Americans who at this very hour patrol far-off deserts and distant mountains. They have something to tell us, just as the fallen heroes who lie in Arlington whisper through the ages. \nWe honor them not only because they are the guardians of our liberty, but because they embody the spirit of service -- a willingness to find meaning in something greater than themselves. \nAnd yet at this moment, a moment that will define a generation, it is precisely this spirit that must inhabit us all. For as much as government can do, and must do, it is ultimately the faith and determination of the American people upon which this nation relies. It is the kindness to take in a stranger when the levees break, the selflessness of workers who would rather cut their hours than see a friend lose their job which sees us through our darkest hours. It is the firefighter''s courage to storm a stairway filled with smoke, but also a parent''s willingness to nurture a child that finally decides our fate.\nOur challenges may be new. The instruments with which we meet them may be new. But those values upon which our success depends -- honesty and hard work, courage and fair play, tolerance and curiosity, loyalty and patriotism -- these things are old. These things are true. They have been the quiet force of progress throughout our history. \nWhat is demanded, then, is a return to these truths. What is required of us now is a new era of responsibility -- a recognition on the part of every American that we have duties to ourselves, our nation and the world; duties that we do not grudgingly accept, but rather seize gladly, firm in the knowledge that there is nothing so satisfying to the spirit, so defining of our character than giving our all to a difficult task.\nThis is the price and the promise of citizenship. This is the source of our confidence -- the knowledge that God calls on us to shape an uncertain destiny. This is the meaning of our liberty and our creed, why men and women and children of every race and every faith can join in celebration across this magnificent mall; and why a man whose father less than 60 years ago might not have been served in a local restaurant can now stand before you to take a most sacred oath. (Applause.)\nSo let us mark this day with remembrance of who we are and how far we have traveled. In the year of America''s birth, in the coldest of months, a small band of patriots huddled by dying campfires on the shores of an icy river. The capital was abandoned. The enemy was advancing. The snow was stained with blood. At the moment when the outcome of our revolution was most in doubt, the father of our nation ordered these words to be read to the people: \n\"Let it be told to the future world...that in the depth of winter, when nothing but hope and virtue could survive... that the city and the country, alarmed at one common danger, came forth to meet [it].\"\nAmerica: In the face of our common dangers, in this winter of our hardship, let us remember these timeless words. With hope and virtue, let us brave once more the icy currents, and endure what storms may come. Let it be said by our children''s children that when we were tested we refused to let this journey end, that we did not turn back nor did we falter; and with eyes fixed on the horizon and God''s grace upon us, we carried forth that great gift of freedom and delivered it safely to future generations.\nThank you. God bless you. And God bless the United States of America. (Applause.)\n\n",

"entities": [["T1","test",[[ 4367, 5040 ]]]] }

spyysalo commented 11 years ago

Thank you for the report. Could you also provide the text intended to be covered by the involved offsets (4367:5040)? I'm getting something unexpected with copy-paste from here, but there may be some encoding issue. Is there perhaps a newline character within the span?

It would also help if you could simplify the case to remove text not related to the problematic span.

Assigning @ninjin .

ghost commented 11 years ago

@spyysalo: Judging from the trace this issue lies deep within the visualiser code rather than with the documentation or the embedding wrapper code. Re-assigning to @amadanmath.

spyysalo commented 11 years ago

The visualizer core is known to be a bit fragile. @ninjin: does the embedding code check its inputs? I think it should.

@TMWong : does this still crash if you simplify the input to remove text other than the problematic span? What would the JSON be in this case?

amadanmath commented 11 years ago

Testing on the live page, for me, the exception is happening only while editing (for example, when editing 5040 into 5041 by deleting 1, which makes the range 4367-504, which should throw an exception).

However, outside of that, 4367-5040 renders wrongly: the highlight disappears. The reason seems to be the span ending on a space; I believe we fixed that (could be wrong.) So I tried it and...

The good news is, it doesn't seem to do so when I tested the same data in master tip. The bad news is, it still does not render perfectly, as the fragment continuation line goes to the supposed edge of the page, not the real edge of the page.

The visualizer core is known to be a bit fragile.

Generously speaking :)

ghost commented 11 years ago

@spyysalo: Check its inputs? Its input are the protocol structures so I think the question you want to ask is "Does the client check what goes over the wire?" and I think we both know that the answer to that question is no. Should it? Yes, preferably. At least after my work on it in November it doesn't go down in flames if you leave out a single field of one of the protocol structures for example.

spyysalo commented 11 years ago

@ninjin : I actually intended to suggest that the embedding code should check the structures. Of course it would also be good if the visualizer core were more robust, but there's no harm in defending in depth against broken inputs.

ghost commented 11 years ago

@spyysalo: Isn't it better to have the checking in one place rather than two? Because the code needed in both places would be identical. I also just want to make clear that the embedding code footprint is about ten lines, the rest is vanilla client.

spyysalo commented 11 years ago

@ninjin : I'd generally argue sanity checking is good even if redundant. However, if this would noticeably increase the size of the embedding code, then maybe it's better to keep that part simple.

TMWong commented 11 years ago

The range 4367-5040 is intended to annotate the text span:

"For everywhere we look, there is work to be done. The state of our economy calls for action, bold and swift. And we will act, not only to create new jobs, but to lay a new foundation for growth. We will build the roads and bridges, the electric grids and digital lines that feed our commerce and bind us together. We''ll restore science to its rightful place, and wield technology''s wonders to raise health care''s quality and lower its cost. We will harness the sun and the winds and the soil to fuel our cars and run our factories. And we will transform our schools and colleges and universities to meet the demands of a new age. All this we can do. All this we will do."

What is more complicated is that the text contains multiple continuous space (they do not show in my previous message, which seem to be trimmed when I post the message). I try to reproduce the text input here, with the irrelevant sentences in the second half removed, and all the space characters replaced with underline characters:

{ "text": "(Inaugural_Address_By_President_Barack_Hussein_Obama:_http://www.whitehouse.gov/blog/inaugural-address/)\n\nMy_fellow_citizens:_I_stand_here_today_humbled_by_the_task_before_us,_grateful_for_the_trust_you''ve_bestowed,_mindful_of_the_sacrifices_borne_by_our_ancestors._\nI_thank_President_Bush_for_his_service_to_our_nation_--_(applause)_--_as_well_as_the_generosity_and_cooperation_he_has_shown_throughout_this_transition.\nForty-four_Americans_have_now_taken_the_presidential_oath.__The_words_have_been_spoken_during_rising_tides_of_prosperity_and_the_still_waters_of_peace.__Yet,_every_so_often,_the_oath_is_taken_amidst_gathering_clouds_and_raging_storms.__At_these_moments,_America_has_carried_on_not_simply_because_of_the_skill_or_vision_of_those_in_high_office,_but_because_we,_the_people,_have_remained_faithful_to_the_ideals_of_our_forebears_and_true_to_our_founding_documents._\nSo_it_has_been;_so_it_must_be_with_this_generation_of_Americans.\nThat_we_are_in_the_midst_of_crisis_is_now_well_understood.__Our_nation_is_at_war_against_a_far-reaching_network_of_violence_and_hatred.__Our_economy_is_badly_weakened,_a_consequence_of_greed_and_irresponsibility_on_the_part_of_some,_but_also_our_collective_failure_to_make_hard_choices_and_prepare_the_nation_for_a_new_age.__Homes_have_been_lost,_jobs_shed,_businesses_shuttered.__Our_health_care_is_too_costly,_our_schools_fail_too_many_--_and_each_day_brings_further_evidence_that_the_ways_we_use_energy_strengthen_our_adversaries_and_threaten_our_planet.\nThese_are_the_indicators_of_crisis,_subject_to_data_and_statistics.__Less_measurable,_but_no_less_profound,_is_a_sapping_of_confidence_across_our_land;_a_nagging_fear_that_America''s_decline_is_inevitable,_that_the_next_generation_must_lower_its_sights.\nToday_I_say_to_you_that_the_challenges_we_face_are_real.__They_are_serious_and_they_are_many.__They_will_not_be_met_easily_or_in_a_short_span_of_time.__But_know_this_America:__They_will_be_met.__(Applause.)\nOn_this_day,_we_gather_because_we_have_chosen_hope_over_fear,_unity_of_purpose_over_conflict_and_discord.__On_this_day,_we_come_to_proclaim_an_end_to_the_petty_grievances_and_false_promises,_the_recriminations_and_worn-out_dogmas_that_for_far_too_long_have_strangled_our_politics.__We_remain_a_young_nation.__But_in_the_words_of_Scripture,_the_time_has_come_to_set_aside_childish_things.__The_time_has_come_to_reaffirm_our_enduring_spirit;_to_choose_our_better_history;_to_carry_forward_that_precious_gift,_that_noble_idea_passed_on_from_generation_to_generation:__the_God-given_promise_that_all_are_equal,_all_are_free,_and_all_deserve_a_chance_to_pursue_their_full_measure_of_happiness.__(Applause.)\nIn_reaffirming_the_greatness_of_our_nation_we_understand_that_greatness_is_never_a_given.**It_must_be_earned.__Our_journey_has_never_been_one_of_short-cuts_or_settling_for_less.__It_has_not_been_the_path_for_the_faint-hearted,_for_those_that_prefer_leisure_over_work,_or_seek_only_the_pleasures_of_riches_and_fame.__Rather,_it_has_been_the_risk-takers,_the_doers,_the_makers_of_things_--_some_celebrated,_but_more_often_men_and_women_obscure_in_their_labor_--_who_have_carried_us_up_the_long_rugged_path_towards_prosperity_and_freedom._\nFor_us,_they_packed_up_their_few_worldly_possessions_and_traveled_across_oceans_in_search_of_a_new_life.__For_us,_they_toiled_in_sweatshops,_and_settled_the_West,_endured_the_lash_of_the_whip,_and_plowed_the_hard_earth.__For_us,_they_fought_and_died_in_places_like_Concord_and_Gettysburg,_Normandy_and_Khe_Sahn._\nTime_and_again_these_men_and_women_struggled_and_sacrificed_and_worked_till_their_hands_were_raw_so_that_we_might_live_a_better_life.__They_saw_America_as_bigger_than_the_sum_of_our_individual_ambitions,_greater_than_all_the_differences_of_birth_or_wealth_or_faction.\nThis_is_the_journey_we_continue_today.__We_remain_the_most_prosperous,_powerful_nation_on_Earth.__Our_workers_are_no_less_productive_than_when_this_crisis_began.__Our_minds_are_no_less_inventive,_our_goods_and_services_no_less_needed_than_they_were_last_week,_or_last_month,_or_last_year.__Our_capacity_remains_undiminished.__But_our_time_of_standing_pat,_of_protecting_narrow_interests_and_putting_off_unpleasant_decisions_--_that_time_has_surely_passed.__Starting_today,_we_must_pick_ourselves_up,_dust_ourselves_off,_and_begin_again_the_work_of_remaking_America.**(Applause.)\nFor_everywhere_we_look,_there_is_work_to_be_done.__The_state_of_our_economy_calls_for_action,_bold_and_swift.__And_we_will_act,_not_only_to_create_new_jobs,_but_to_lay_a_new_foundation_for_growth.__We_will_build_the_roads_and_bridges,_the_electric_grids_and_digital_lines_that_feed_our_commerce_and_bind_us_together.__We''ll_restore_science_to_its_rightful_place,_and_wield_technology''s_wonders_to_raise_health_care''s_quality_and_lower_its_cost.__We_will_harness_the_sun_and_the_winds_and_the_soil_to_fuel_our_cars_and_run_our_factories.__And_we_will_transform_our_schools_and_colleges_and_universities_to_meet_the_demands_of_a_new_age.__All_this_we_can_do.__All_this_we_will_do.\nNow,_there_are_some_who_question_the_scale_of_our_ambitions,_who_suggest_that_our_system_cannot_tolerate_too_many_big_plans.__Their_memories_are_short,_for_they_have_forgotten_what_this_country_has_already_done,_what_free_men_and_women_can_achieve_when_imagination_is_joined_to_common_purpose,_and_necessity_to_courage.__What_the_cynics_fail_to_understand_is_that_the_ground_has_shifted_beneath_them,_that_the_stale_political_arguments_that_have_consumed_us_for_so_long_no_longer_apply.",

"entities": [["T1","test",[[ 4367, 5040 ]]]] }

In this case the annotation works, although not exactly the range I want. But once the underline characters are replaced with space again, the annotation fails.

spyysalo commented 11 years ago

@TMWong : thank you for the further details. @amadanmath suggested that this might relate to the span beginning or ending on a space, and that the issue could potentially be fixed in the latest master. Could you please try if adjusting the edges of the span or setting up the latest development version from https://github.com/nlplab/brat/ would help?

amadanmath commented 11 years ago

Be aware that you can format as code on GitHub if you indent with four spaces, in which case you don't get the normal text treatment of whitespace.

Normal text: double space between words

Code: double  space  between  words

Having said that, I can confirm there is an error happening. I will investigate...

amadanmath commented 11 years ago

OK, I think I know what is happening - and in an ironic twist, pretty much explained it in the last comment. :) The problem comes from the double spaces. In an XML document (and thus also inside our SVG), double space is effectively the same as a single space. Thus, the string

"a  b"

has four characters, while the SVG text element I measure,

<text>a  b</text>

only has three. To find out the width of the string, we measure the right position of the last character, which would be character number 3 - but there are only characters 0, 1 and 2 ('a', ' ', 'b'), thus the error.

SVG API is horribly impossible to find (believe or not, but MSDN is the best resource!), but Googling around I found getNumberOfChars(), so I think it will work if I replace the assumed number of characters with what the text element is returning. Cheers and thanks for the report.

spyysalo commented 11 years ago

@amadanmath : does this also address #832?

amadanmath commented 11 years ago

does this also address #832?

No; marked spans (like in search results by text) still use number of characters. Thus, with our backend (as opposed to embedding), there is a rare but possible scenario where:

You have a span (or a fragment) that includes double space
You have a focus argument (or a text search result) containing a text range in such a span

The hack still protects us against that.

Maybe if we removed all "redundant" space characters from the marked string and then deducted the difference in length in the marked span case calculation...?

TMWong commented 11 years ago

Using the latest codes I can now at least get the visualization output. However, a strange error occurs that two different ranges starting with the same position: will result in the same range of visualization:

It works correctly when removing any one of them.

nlplab / brat

Error: IndexSizeError: DOM Exception 1 Error: Index or size was negative, or greater than the allowed value. #1009