# [Sandro Hawke] Meeting starting soon
# [Sandro Hawke] https://credweb.org/agenda/20180530.html
# [Cheryl Langdon-Orr] So may as well start...
# [Cheryl Langdon-Orr] I am unwilling to scribe at 0300 laying down in my bed
# [Cheryl Langdon-Orr] Try 2000 UTC
# [Sandro Hawke] PROPOSED: Approve minutes of last week https://credweb.org/minutes/20180523.html
# [Sandro Hawke] +1
# [Reto Gmür] sure
# [Cheryl Langdon-Orr] Approve .+1
# [rhiaro] +0 on minutes, I wasn't there so can't validate
# [Sandro Hawke] RESOLVED: Approve minutes of last week https://credweb.org/minutes/20180523.html
# [Amy Zhang] +1 (and I made a small pull request adding myself as a scribe to last week)
# Sandro Hawke says: there will be a monday meeting on inspection next week
# Sandro Hawke says: the decision on meeting date will be made in the next days
# [Sandro Hawke] Vocab example: { <example.com> :domainCredibility 0.80 }
# Sandro Hawke says: this would probably do more harm than good but still interesting trying
# Jon Udell says: problem about just giving rating based on political affiliation
# Ed Bice says: : This cannot be decoupled from incentive. Incentive drives from fact that information is structured
# Ed Bice says: credibility only partially depends on reputation
# Ed Bice says: structure that allows other to verify.
# Sandro Hawke says: there are lot of possible use cases, one could make a decentralized version of these. What makes it trustworthy is that you tell who you trust. Building a reputation graph.
# Reto Gmür says: it's not centralized
# Reto Gmür says: I don't see a need for centralisation in this use case, anyone can publish who they trust in certain areas, and we wouldn't have to mandate any particular algorithm
# Ed Bice says: I feel like we should be aware that with open data project one can easily aggregate all the data. An Ad-tech company could do exactly that. It's a way to end run [missed]
# Ed Bice says: I think the goal is to showing the proof. Reputation would
# Ed Bice says: I am supporting the work on reputation. If Musk-like site was going to happen, we should be aware that with an open data project someone can easily aggregate all of the data generated against these standards, and there's a rumour that there are discussions happening through NII with a big ad tech company about supporting that, which to me.. my point is that the work we're doing to develop the indicators are a way.. it's not an either/or, it's a way to run the reputation piece and say how were these various indicators structures. What is the line of arguement that the first time user of the site, public use of these standards is making about this and how am I evaluating that or how is that unit being regarded .. I think this is a lot of the spirit of building these indicators is to create a way to show the proof and indicators are meant to stitch together a fabric of that. In most cases reputation will the be the thing. Curious to get @Amy Zhang's response
# Sandro Hawke says: : The same open data can be used in different algrithms
# Sandro Hawke says: : everybody can come along and use the data
# [Ed Bice] agree with this Sandro - though we would imagine that thiel and musk would presumably filter against the networks they trust.
# Reto Gmür says: there is some degree of decentralisation so you're not getting thes ame results. People use different rules of trust and have differenta lgorithms. Somebody may say no everything that isn't scientific doesn't get considered by my computations, and others may use other criteria
# ... I choose my bubble more consciously
# Sandro Hawke says: : it could be done on your own machine
# Sandro Hawke says: : there are two dimension of decentralization and the root of trust
# Reto Gmür says: [missed] technological decentralisation vs social decentralisation. We don't disagree, anyone can put this information on the web and we can aggregate it on our own machines
# Reto Gmür says: : logical and technological decentralizdation
# Sandro Hawke says: : I listed two reasons why this would be bad: human negativity (bringing down the godo people) and coerced statements.
# Sandro Hawke says: : partisan media sites would be more likely to be open on who you support. Non-partisant may not want to be open or think they have to lie about who they support, peer pressure, pressure by influencee/employers
# Jon Udell says: : If I already trust XY what''s the benefit of the statements
# Sandro Hawke says: : If I trust XY I can trust the sources recommended by XY
# Jon Udell says: : valuing endorsments depending on where the money comes from
# Sandro Hawke says: : one way is to prohibit negative rating
# Reto Gmür says: the negativity bias can partly be addressed by the algorithm. Eg. for movies, they don't look at the content of reviews but at the number of reviews or how contraversial they are
# ... you can use other criteria to .. we don't mandate the algorithm, but there are good arguements that algorithms can be used that at least alleviate the problems caused by some kind of bias
# Sandro Hawke says: I wonder if some of that can be surfaced with a few more properties, like naming taht I think this site has a conflict of interest or financial interest in this topic
# Reto Gmür says: I think it's very much a complement to claim reviews ?? narrow quality.. there are reviews of individual articles, and reviews of sources of articles. Then of course fi you know a source which has all good articles according to your computation, then it has some influence on the source, and obviously the other way around. A credible source is defined as a source of credible articles
# [Reto Gmür] (deleted)
# Sandro Hawke says: : can we find people that are alraedy working on this problem
# Sandro Hawke says: There may even be academic conferences or workshops on this subject, I haven't looked
# Reto Gmür says: : we could distinguish between the more fundamental work which makes the web a place where you know why to believe some information. Changes the architecture in such aw ay that you can have good reason to trust information on the web
# ... not just because of the fact it's on the web, that will never be a good reason to trust inforation. But things like transparency and reputation. I would call this area of activity of standardisation as "credibility by default"
# ... this would be the more long term proactive field of activity. Shorter term is more reactive, about how to find indicators when should a red flag on the moderator screen pop up
# ... or a claim reviews which are an incredible use that don't really scale. First you have the false claim spreading, then the review.. on one hand we have to make the future building fireproof. On the other hand we have to have firefighters and smoke detectors
# ... firefighters are the factcheckers, smoke detectors are the indicators, and I think transparency and reputation is the fireproof building
# Ed Bice says: number f people of people who scribes is an indicator. I think a big peace we bring into this is the contextuality, reputation as a contextually contingent attribute
# [Sandro Hawke] Vocab example: { <example.com> :domainCredibility 0.80 }
# Sandro Hawke says: : the domain of credibility is about havin the context. You could have a vocabulary that allows expressing the credibility on a filed. For examplesome people think the guardian is credible about everything except what relates to julian assange
# Ed Bice says: : there is cristal clear examples [...] meaning of picture of plane in Hudson river depends on context [...] even withou knowing the langauge one can assert credibility
# Sandro Hawke says: : people I trust also correctly represent their crendentials
# ... you have to let people do that tnemselves
# [Sandro Hawke] honesty + competence --- self report
# Reto Gmür says: if you trust someone is honest and they say they are a genious in this domain, then you trust waht they say about that domain. There is the case that I really don't trust someone to be intellectually honest or able to observe their own competences in a good way, but nevertheless they're a genius at cooking.. don't trust him if he says osmething unless it's a recipe
# Sandro Hawke says: : from engineering perspective adding the domain makes it much more complex
# Ed Bice says: : we just need to have the filed, we leave it to algorithms if they do domething about it
# Sandro Hawke says: : you have to keep track of who said it
# Amy Zhang says: : what isthe reson to have a scalar?
# Sandro Hawke says: : this allows math, like transitivity
# Amy Zhang says: : people scale differently
# Sandro Hawke says: : two possible answers, specifying in spec, other to put scientific probaboloty
# Reto Gmür says: there are studies showing... if you have a study asking people how happy they are ona scale from 1 to 10, people from switzerland position themselves in the middle, and people in spain on the extreme. No matter what the question, it's a cultural thing. Even adding terms doesn't change anything or makes it worse. In some cultures you just don't use some extreme terms. The scientific thing, how would you default weight a proposition in a computation
# [Sandro Hawke] 3rd option: algorithm re-scales per-source
# ... it would be quite well defined but it wouldn't change anything. Some people have more tendancy to be moderate and some more extreme. If they are just among peers who do the same it will neutralise itself, but if there are these cultural differences then I think they have to be deferred to the algorithm that's more sophisticated will find out that use very strong ratings or you are a moderate and are always around the middle and so that effectively 5.2 on the scale means the same as a 9 from another person
# Reto Gmür says: we will always have differene in how people rate
# Sandro Hawke says: : fact checkers could mathematically compute based on ration ontrue sattement
# Reto Gmür says: what prevents me as a fake news site provider to just put a whole lot of true statements to water down my real intention?
# [Sandro Hawke] threat technique: publish 99% benign facts, 1% nasty truths
# ... makes it much more important what the fact checker chooses to fact check. My credibility becomes very high even if I also present some very dubious story
# Jon Udell says: : a lot of verdict is not specified numerical. A lot of claim reviews have just qualitative results
# Sandro Hawke says: : ...
# Jon Udell says: : if some statements are inconclusive other say "it depends" how would this project to binary
# [Sandro Hawke] edbice: this binary vs linear value is a testable hypthosis
# Sandro Hawke says: : if there are numerical values you see often, the you can start comparing
# [Amy Zhang] that would be me @Reto Gmür not @An Xiao Mina
# Sandro Hawke says: : i couldn't give a number to a movie, but if I see a lot of other ratings I can
# Farnaz Jahanbakhsh says: : [...]
# Sandro Hawke says: : there are different algorithms
# Farnaz Jahanbakhsh says: how does transitive trust work with ranked lists?
# Jon Udell says: : would be good to have samples of diffrent source and correlate
# [Sandro Hawke] possible study -- run this on claimreview crawl
# Reto Gmür says: of course we can defer a lot to the algorithms but I think we should find as ismple as possible but flexible standards for publishing this data
# ... one thought is to have next to the actual rating which indicates something about likelihood of truth, to have something like a weight. The same goes.. I have this problem when writing reviews on any review site, how good was it? In one situation I came 30 seconds before the shop closed and they don't let me in, so I give them a bad review. People have a bad day.. something completely forgivable
# ... the other situation is they sold me something broken, made wrong promises, they ripped me off again.. also a bad rating, but has a much bigger weight
# ... and thes ame goes for positive ratings. You get really good support, or they're just doing what they're supposed to do in a friendly manner. Both are 5 stars, but one 5 star is good to tell people about it, but the other is just nothing to complain
# Amy Zhang says: it doesn't work for coordinated activities
# Giovanni Luca Ciampaglia says: possibly Jen Golbeck is working on that stuff
# Giovanni Luca Ciampaglia says: : there is a lot in research worth digging into
# Sandro Hawke says: : one other space that might be relevant: there are companies that do sentiment analysis on social media
# Giovanni Luca Ciampaglia says: : there are also companies that detetctwhen you are being shamed [?]. Public shaming on twittter. The companies try to [...], there was a problem with [...] depending on context
# Sandro Hawke says: : how bad would it be to launch something in this space
# Giovanni Luca Ciampaglia says: :[sorry didn't get],
# Giovanni Luca Ciampaglia says: our lab got attacked by a social media campaign for studying memes
# Sandro Hawke says: : john mentioned the idea of a seeing how claim reviews can be aligned. On twitter you could have indiduals post therir ratings
# Sandro Hawke says: : tools could aggregate this. All the people I follow, default as endorsed
# Giovanni Luca Ciampaglia says: : aggregating on this data would reflect split, controversial views would neutralize them out. BUt you could have recommendations based on similarity, people also like this...
# Giovanni Luca Ciampaglia says: : recommender system is like collaborative filtering
# Reto Gmür says: isn't twitter already doing this? It has little to do with trust and credibility
# Sandro Hawke says: right, different algorithm with different inputs. Entertaining rather than credible
# [Sandro Hawke] recommender system: recommend as credible (not as entertaining)
# Reto Gmür says: now you mention the users of twitter.. users asserting the reptuation as users.. I think it's important to keep the truth and credibility dimension separate from the common 'like' dimension
# ... this might be something which changes the architecture towards more credibility by having these separate
# ... something can be entertaining
# Sandro Hawke says: absolutely, I think tha'ts how social media screwed us over over the last 10 years
# Reto Gmür says: that's why before when you said the way the internet is doing this, and the question weight vs actual statement, that maybe exactly where the problem lies. Not the only root. Just because the internet has been doing this in a certain way so far is not a particularly good rason to keep it this way
# Sandro Hawke says: I'm imagining a game, it used to be popular on twitter to use the #ff followfriday hashtag, who you recommend everyone else follow. To do... credibility tuesdays.. not quite the same,b ut you can call out the folks who are really credible sources, and who are not. If people could easily use that data automatically, just the fact that you put it on twitter would be then used by these systems consuming that and building a network out of it. Maybe there's something fun and useful there
# Sandro Hawke says: : a hashtag could be use to call out the folk who are credible sources. If people could aitomatically use that data there could be something fun and usefuk there
# Reto Gmür says: LinkedIn lets you say someone is proficient in a particular domain, we could look at that
# Sandro Hawke says: : Linkedin endorsement have some strange artifacts