MachineVisionUiB / machinevision

We are developing a database to map and interpret the representations and uses of machine vision technologies in digital art, computer games and narratives such as science fiction novels, movies and creepypasta.
http://uib.no/en/machinevision
4 stars 0 forks source link

Tag consultations #15

Closed jilltxt closed 3 years ago

jilltxt commented 5 years ago

When in doubt about how to tag a work in the database, please describe the issue and your preliminary choice of tagging here. All other team members should consider the issue. Leave a thumbs up icon on the comment if you approve, or respond to the commend for further discussion. The goal is to achieve consensus and develop a more precise tagging system.

LindaKairus commented 5 years ago

I am working on 10.000 Moving Cities – Same but Different, VR (Virtual Reality) by Marc Lee: http://marclee.io/en/10-000-moving-cities-same-but-different-vr/

It is a VR installation where the user gets a VR set. You can choose pretty much any city in the world and then a city space is rendered with real time social media streams on the facades of buildings. So you see YouTube Videos, twitter feeds, Flickr images etc from the city you chose. So obviously the tech used is VR. But I was wondering if the tech used to create the social media images (camera, camera phone, webcamera) should be chosen in the category referenced technology? Or should I just leave it out?

LinnHeidi commented 5 years ago

Did you figure this out? I saw you only added VR in the work. I don`t think it is necessary to tag the others.

I was wondering about this one: https://machinevision.elmcip.net/content/digital-scores-after-nic%C3%A9phore-ni%C3%A9pce

I can not think of themes or technology? should we keep it or? It is the first photograph ever taken rendered into code on a print.

LinnHeidi commented 5 years ago

I am also not sure what technology this one is using: https://machinevision.elmcip.net/content/rememberimagine

LinnHeidi commented 5 years ago

https://machinevision.elmcip.net/content/parametric-expression HELP! What/whom is being viewed?

ragsol commented 5 years ago

I wonder: What is relevant/what is actually "machine vision"? (lol) For instance, I won't log all VR games - there has to be something "more" (that I don't really know what is). And for cellphone cameras, do we log all cases of selfies being taken? Just the ones where they are put in a greater "system" of MV, e.g. using filters? Just if referenced X number of times? We probably can't make a rule for the fringes, but it's useful to discuss :)

ragsol commented 5 years ago

Perhaps we need "comic book" as a type? Now I can only tag it as "narrative" or e-lit.

jilltxt commented 5 years ago

@LinnHeidi about Remember/Imagine: I took a look at this piece, and it's tricky. I think we can put quite a few technologies referenced, though, camera, filtering, image generation, satellite images? Tricky because it's definitely about machine vision but sort of so poetically it's almost abstract. I also struggled to figure out the sentiment (which I think you hadn't tagged?) I wanted to put "wonder" - the piece is sort of like Obstfelder's Jeg ser, jeg ser. I ended up just putting exciting. Not sure. Do you agree? Also, what do we do about situations?

LindaKairus commented 5 years ago

I will need a second opinion of my interpretation of U: https://machinevision.elmcip.net/content/u I have the passwords for the film if someone has time to look at the whole thing with me 37min

ragsol commented 4 years ago

Is thermal scanning of human bodies 'body scan' or 'non-visible spectrum'? (as seen in Minority Report)

jilltxt commented 4 years ago

I'd say non-visible spectrum. Body scan is using technologies to see things that are normally hidden, and we specifically state that an app diagnosing a melanoma wouldn't be. But we need to change the definition of non-visible spectrum in the wiki so it's not just low-light, but also heat etc. It was useful being able to go back to the recording of our discussion and find the bit about infrared (now renamed non-visible spectrum). Also, it's GREAT that you got some Minority Report situations in the database!

ragsol commented 4 years ago
jilltxt commented 4 years ago
LinnHeidi commented 4 years ago

I am retagging https://machine-vision.no/content/cowboy-bebop-jamming-edward-ed-hacks-satelite and I am wondering what to do when the Technology is obviously also a character (A hacked satellite that draws with a laser onto Earth surface because he misses humans)

In the situation I have added 2 characters: The hacker (Radical Ed) and the Satelite (MPU), and I also added Satellite Images as a technology, just because I felt we needed a tech bit, but do we?

jilltxt commented 4 years ago

The satellite is clearly a character, yes. But if we don’t also log it as tech we lose the data that it is satellite - so I think log it twice, as a char and as tech?

LinnHeidi commented 4 years ago

Yep, that`s what I figured too! Thank you!

ragsol commented 4 years ago

Isn't it possible to export tech and characters together, so that the distinction really doesn't need to be made? I think the idea was to treat both tech and characters (and entities) as if they are actors. We lose the ability to tag for characteristics for the character, though, if it's just in tech. Is it possible to add characters to tech as well? (I'm not sure if this post is understandable, but there might be some discussion material in here :))

jilltxt commented 4 years ago

I'm a bit confused! Yes, we can definitely export tech and characters together, but they have different characteristics. I suppose we could add technologies to the characters, so that if the character is a machine we could say which machine vision technologies the character "is" or "has"? I feel like the fact that the technology is a character actually says a lot about its agency, too. Hm.

LindaKairus commented 4 years ago

I just re-taged this one: THE GOOGLE TRILOGY – 3.The Driver and the Cameras and I was wondering if I should also add Google as a company and would the verb then be owning, failing because their policies of anonymizing people on the StreetView has failed or is this to detailed and not relevant. Somehow at least I think it is relevant that a company is owning the tech...

LindaKairus commented 4 years ago

I am not quite sure when to use Using and Referencing Technology. If the User is using the tech it is clear, then it is usually an interactive installation where the user is exposed to machine vision. But then in the artificial neural network generated images, or e.g. in the THE GOOGLE TRILOGY – 3.The Driver and the Cameras the artist is using the machine vision is it then used or referenced, because in the current system this becomes a bit more vague because the artist can be an active entity.

jilltxt commented 4 years ago

I am not quite sure when to use Using and Referencing Technology. If the User is using the tech it is clear, then it is usually an interactive installation where the user is exposed to machine vision. But then in the artificial neural network generated images, or e.g. in the THE GOOGLE TRILOGY – 3.The Driver and the Cameras the artist is using the machine vision is it then used or referenced, because in the current system this becomes a bit more vague because the artist can be an active entity.

An image generated by a neural network should be tagged for Technology Used - because machine vision was used you produce the work. In many cases you’d also tag it for Technology Referenced, but only if the images or the way they’re exhibited clearly show that they’re ABOUT machine vision, so if it’s pretty obvious or made a point of. I can imagine some neural network images where it’s really not emphasized.

jilltxt commented 4 years ago

owning

Yes, use Corporation in the Generic Entities section. I think it’s best not to specify Google but it’s definitely relevant to put Corporation owning, failing.

ragsol commented 4 years ago

The issue is with the separation of Character and Technology. Characters are for when we want to catch more nuance than just a generic role ('User') or a generic technology ('Drone'). Let me try to flesh out part of the problem with a recent example: Tom Clancy's Ghost Recon: Wildlands. In the drone situation, players use a drone to gather intel and possibly to kill. When I write 'Killing', I do so on the Characters, but the drone is part of the assemblage. Do I then write 'Killing' on both?

For Linn Heidi's case, it's a question about how much agency we allow an agent. Right now, if we deem them 'agential enough', we will create them as a Character, if not, they are Technologies. There is potential for discussion here as well, but if we say this is how we want to do it, wouldn't it stack with adding the Character-Drone as a Technology-Drone?

Possible solutions (perhaps there are more?)

So, my initial response was to highlight that it seems as if we always have to export Characters and Technologies together, because doing anything else leaves us missing out on important data. If this is the case, we can't double-tag Characters and Drones, I think. I hope this makes sense! :)

jilltxt commented 4 years ago

In order to analyse the assemblage of characters, technologies and actions I think we need to register them separately with the same verbs. To test it, I tried making a very simple Gephi network graph just of the two actors in this situation - the Player and the Drone. (And then afterwards I looked at the situation in the database - not sure how to correlate)

Here is what I imagine the data might look like exported as a spreadsheet (I just typed this into Excel - the verbs might be different)

Screen Shot 2019-11-12 at 11 03 52

Then importing this to Gephi the node table would look like this:

Screen Shot 2019-11-12 at 11 11 01

and the edge table like this:

Screen Shot 2019-11-12 at 11 01 06

And the network would look like this (coloured by type, and sized according to degree, with a ForceAtlas2 algorithm)

Screen Shot 2019-11-12 at 11 00 05

I think this expresses the assemblage really well. I'm not exactly sure how, but I think we can probably figure out a way to do this for each situation but in a big graph with many situations, or we could do an overall analysis of which actors (characters, techs or entities) tend to work together, or cluster around different verbs. Maybe we could even show this kind of mini diagram in individual situations on the database.

I can't see many cases where we wouldn't want to export characters with technologies and entities? So it makes sense to register an assemblage by simply using the same verbs for multiple actors? There may also be cases where a drone does x, y and z and a character does x and y but not z, and that could be interesting.

If you want to play with this (very simple!) Gephi file it is here: https://universityofbergen.sharepoint.com/:u:/r/sites/MachineVision/Shared%20Documents/Database/test-of-character-and-tech-with-verbs.gephi?csf=1&e=MZ3CC3

LinnHeidi commented 4 years ago

Not sure if we should keep this one: https://machine-vision.no/content/digital-scores-after-nic%C3%A9phore-ni%C3%A9pce

jilltxt commented 4 years ago

@LinnHeidi, probably delete? Or just leave but not add to? I'm not quite sure what the machine vision technology would be here, or what what a situation would be - although I do see that it has to do with digitalisation of visual stuff.

ragsol commented 4 years ago

If everything I see is through an AI, do I then also tag for AR? The interface is obviously AR for a human, but for the AI, I guess it's normal..? =) Here's the work: https://machine-vision.no/content/observation

jilltxt commented 4 years ago

Oy. I have no idea. I guess it's not really AR since that's just the way the machine sees it?

jilltxt commented 4 years ago

It doesn't matter much whether it's AR or not until you want to assign a verb to the tech. At the point where you want to say "AR does X" in this situation, I think you'll know whether or not it counts as AR.

ragsol commented 4 years ago

If the player character/avatar is a machine, is it database material? Like, if Detroit didn't have the investigation overlay, could it still be logged because the player characters are androids and the world is seen through them even if it's not from a first person perspective? It seems like a stretch, but at the same time, we would probably log this if it was in a book or a film..? (Edit: See this attempt at a situation for reference https://machine-vision.no/content/detroit-become-human-ai)

jilltxt commented 4 years ago

@ragsol, I looked at the situation - with the current description it doesn't specifically deal with machine vision. We could rewrite the description to say that the androids are represented as seeing more information about the world than we are or something like that. Can you imagine a way of rewriting the description to make this about machine vision? If not, I think it probably shouldn't be a separate situation.

LindaKairus commented 4 years ago

Work: https://machine-vision.no/content/narciss MV-situation: https://machine-vision.no/content/narciss-self-analysing-machine

In this work/situation I have divided the tech into two entries: 1) The actual technology camera, machine learning and object recognition is one assemblage that is actually used in this installation. I have merge them together with verbs though I could probably separate them, at least the camera... should I do this?

2) Then I have another tech entity that is the more speculative General AI and this is where I put verbs like self-reflecting end Mirrored because the machine is looking at it self trying to figure out what it is. I did not use a character here because I do not think it has any characteristics that are relevant. I also did not create a generic entity like Machine because I am not sure on what level we want to catch this and somehow my gut feeling that this is more on a speculative technology level. Is there other opinions?

Further both the Creator and the Observer are part of the meaning making process. Here I used the Generic entity of User instead of Observer even if there is not really a interaction going on between the user and the machine. I am just hesitating to create another entity such as observer. But I think this is a fringe case for us to define what we mean with User. Can I use User or create a new generic entity called Observer?

jilltxt commented 3 years ago

This has been inactive for a while, and I don't think we have the same need for it, so I am closing the thread.