Reliable Methods of Judgment Aggregation

StephanHartmann†,GabriellaPigozzi‡,JanSprenger§

Abstract

Theaggregationofconsistentindividualjudgmentsonlogicallyinter-connectedpropositionsintoacollectivejudgmentonthesameproposi-tionshasrecentlydrawnmuchattention.Seeminglyreasonableaggrega-tionprocedures,suchaspropositionwisemajorityvoting,cannotensureanequallyconsistentcollectiveconclusion.Theliteratureonjudgmentaggregationreferstosuchaproblemasthediscursivedilemma.Inthispaperweassumethatthedecisionwhichthegroupistryingtoreachisfactuallyrightorwrong.Hence,thequestionweaddressinthispaperishowgoodthevariousapproachesareatselectingtherightconclusion.Wefocusontwoapproaches:distance-basedproceduresandBayesiananaly-sis.Undertheformerwealsosubsumetheconclusion-andpremise-basedproceduresdiscussedintheliterature.WhereaswebelievetheBayesiananalysistobetheoreticallyoptimal,thedistance-basedapproacheshavemoreparsimoniouspresuppositionsandarethereforeeasiertoapply.

1Introduction

Membersofagroupoftenhavetoexpresstheiropinionsonseveralproposi-tions.Examplesareexpertpanels,legalcourts,boards,andcouncils.Oncethemembershavestatedtheirviewsontheissuesintheagenda,theindividualjudgmentsneedtobecombinedtoformacollectivedecision.Theaggregationofindividualconsistentjudgmentsonlogicallyinterconnectedpropositionsintoanequallyconsistentgroupjudgmentonthesamepropositionshasrecentlydrawnmuchattention.Thedifficultyliesinthefactthatthereisnogeneralagreementuponwhichproceduretouse.

Inthispaperweevaluateandcomparethemethodsproposedsofarinthelit-eraturewithaBayesianapproachtojudgmentaggregation.Theassessmentcriterionweemployisreliability.Weassumethattheresultingcollectivejudg-mentisfactuallyrightorwrong,andwecomparetheproceduresintermsofhowreliabletheyareatselectingtherightsocialdecision.Inparticular,wepresentresultsconcerningthereliabilityofseveralmethodstoaggregatecon-flictingindividualjudgmentsintoaconsistentgroupconclusion.Thefirstgroupofmethodsaredistance-basedprocedures,amongthemthemajorityfusionop-erator[11].Fusionoriginatesfromcomputerscience,wheretheproblemofcombininginformationfromequallyreliablesourcesarisesinseveralcontexts[5].ThesecondmethodisafullBayesiananalysisoftheunderlyingdecisionproblem.

Thecombinationoffinitesetsoflogicallyinterconnectedpropositionshasbeenrecentlyinvestigatedintheemergingfieldofjudgmentaggregation[9].Ajudg-information:StephanHartmann,TilburgCenterforLogicandPhilosophyof

Science,TilburgUniversity,NL-5037ABTilburg,S.Hartmann@uvt.nl

‡Contactinformation:GabriellaPigozzi,ComputerScienceandCommunicationsDepart-ment,UniversityofLuxembourg,L-1358Luxembourg

§Contactinformation:JanSprenger,TilburgCenterforLogicandPhilosophyofScience,TilburgUniversity,NL-5037ABTilburg,J.Sprenger@uvt.nl.

†Contact

1

Judges1,2,3Judges4,5Judges6,7MajorityAYesYesNoYesBYesNoYesYesCYesNoNoNoTable1:Anillustrationofthediscursivedilemma

mentisanassignmentofyes/notoaproposition.Theproblemisthataseem-inglyreasonableaggregationprocedure,suchaspropositionwisemajorityvoting,cannotensureaconsistentcollectiveconclusion.

Hereisanillustration.Acourthastomakeadecisiononwhetherapersonisliableofbreachingacontract(representedbyapropositionC,alsoreferredtoastheconclusion).Thejudgeshavetoreachaverdictfollowingthelegaldoctrine.ThisstatesthatapersonisliableifandonlyifshedidanactionX(representedbypropositionA,alsoreferredtoasthefirstpremise)andhadcontractualobligationnottodoX(representedbypropositionB,alsoreferredtoasthesecondpremise).Thelegaldoctrinecanbeformallyexpressedastheformula(A∧B)↔C.EachmemberofthecourtexpressesherjudgmentonA,BandCsuchthattherule(A∧B)↔Cissatisfied.SupposenowthatthecourthassevenmembersmakingtheirjudgmentsaccordingtoTable1.Weseethat,althougheachjudgeexpressesaconsistentopinion,propositionwisemajorityvoting(consistingintheseparateaggregationofthevotesforeachofthepropositionsA,BandCviathemajorityrule)resultsinamajorityforAandamajorityforB,butinamajorityfor¬C.Thisisclearlyaninconsistentcollectiveresultasitviolatestherule(A∧B)↔C.Theparadox(calledthediscursivedilemma)restswiththefactthatmajorityvotingcanleadagroupofrationalindividualstoendorseanirrationalcollectivejudgment.Clearly,therelevanceofsuchaggregationproblemsgoesbeyondthespecificcourtexampleanditappliestoallsituationsinwhichindividualbinaryevaluationsneedtobecombinedintoagroupdecision.

Twoescape-routestothediscursivedilemmahavebeensuggested:thepremise-basedprocedure(PBP)andtheconclusion-basedprocedure(CBP).AccordingtoPBP,eachjudgevotesoneachpremise.Theconclusionistheninferredfromtherule(A∧B)↔CandfromthejudgmentofthemajorityofthegrouponAandB.IncasethejudgesoftheexamplefollowedthePBP,thedefendantwouldbedeclaredliableofbreachingthecontract.Ontheotherhand,accordingtotheCBP,thejudgesdecideprivatelyonAandBandonlyexpresstheiropinionsonCpublicly.ThejudgementofthegroupistheninferredfromapplyingthemajorityruletotheindividualjudgmentsonC.Intheexample,contrarytothePBP,theapplicationoftheCBPwouldfreethedefendant.Moreover,noreasonsforthecourtdecisioncouldbesupplied.

Inthispaper,westudyfurtherpropertiesoftheinformationfusionprocedurewhichtakesamiddlepositionbetweenPBPandCBP.Aboveall,weaddressthequestionhowgoodanaggregationprocedureisatselectingtherightconclusion.1ThebehaviorofthefusionprocedurewillbecontrastedwiththePBPandtheCBPthatwerestudiedbyBovensandRabinowicz[2]andbyList[7,8].Fur-thermore,weapplyBayesianconditionalizationtothegroupdecisionproblem.

[4]foraninvestigationofaggregationproceduresintermsofreliabilityinselecting

therightsituation,i.e.premisesandconclusion.

1See

2

ItisshownthattheBayesianapproachenjoystheoreticaloptimalityandhighflexibility.Inparticular,wecancombineitwithanysetofpriordistributionsandutilitymatrices.Ontheotherhand,itrequiresacostlycomputationofaposteriordistributionwhereboththepriordistributionandthecompetenceofthevotershavetobemadeexplicit.Theserequirementsmaybehardtomeetinmanypracticalapplications.Finally,wecomparethedistance-basedprocedurestotheBayesiananalysis.

Thepaperisstructuredasfollows:InSection2,wedescribethefusionprocedureandshowthatitisanelementofacontinuumofdistance-basedprocedureswhichalsocontainsPBPandCBP.Section3comparesthesethreeapproachesintermsoftheirreliabilitiesatselectingtherightconclusion.AfullBayesiananalysisofagroupdecisionproblemisprovidedinSection4.Section5concludesandsketchesfurtheropenquestions.Finally,theappendixcontainstheproofsandthecalculationdetails.

2

2.1

Distance-basedprocedures

Introduction

Asshownin[11],theapplicationofafusionoperatortojudgmentaggregationproblemsallowstodefineconsistentgroupdecisionsandtoavoidparadoxicaloutcomeswithouthavingtochoosebetweentwopossiblyconflictingprocedureslikethePBPandCBP.Thissubsectionsummarizestheapproachandtheresultsof[11].Thereaderisreferredtothatpaperformoredetails.

Oneofthekeypointsintheliteratureoninformationfusionisthattheaggrega-tionoffinitesetsofpropositionssatisfyingsomeconstraintsdoesnotguaranteeacollectivejudgmentsatisfyingthesameconstraints.Onewaytoovercomethisproblemistorestrictthespaceofthepossiblesolutionstothesetoftheadmissiblesituationsonly,i.e.tothosesetsofpremisesandconclusionthatsatisfytheconstraints.Then,thefusionoperatorselectsoneoftheseconsis-tentsituations,namelythe(possiblynotunique)elementthatminimizesthedistancetotheactualindividualinputs.

Toillustratehowthemajorityfusionoperatorworks,weapplyittothecourtexample.WehavetoformajudgmentonthesetofpropositionsX={A,B,C}withtheconstraint(A∧B)↔C.Hence,thereareonlyfourconsistentsitua-tions:

S1={A,B,C}=(1,1,1)S2={A,¬B,¬C}=(1,0,0)S3={¬A,B,¬C}=(0,1,0)S4={¬A,¬B,¬C}=(0,0,0)

Inthisterminology,Aisidentifiedwitha1and¬Awitha0.InagroupofNpersons,therearen1personsendorsingthesituationS1(i.e.theyjudgeA,BandCtobetrue),n2personsendorsingS2,andsoon.Hence,n1+n2+n3+n4=N.Onpainofindividualirrationality,everymemberofthegrouphastoendorseexactlyoneofthesesituations.Inprinciple,theequationsin(1)involveanabuseofnotationbecausethesituationsrefertostatesoftheworldaswellastoelementsinavectorspacethatbeardistancerelationstothesubmitted

(1)

3

Judges1,2,3Judges4,5Judges6,7AverageHammingdistanceHammingdistanceHammingdistanceHammingdistancetotototoS1S2S3S4(componentwise)(componentwise)(componentwise)(componentwise)A1105/72/72/75/75/7B1015/72/75/72/75/7C1003/74/73/73/73/7TotalS1S2S3—8/710/710/713/7Table2:Thedistance-basedfusionoperatorintheoriginalexampleoftable1.judgments.Nonetheless,theintendedmeaningof“S1”willalwaysbeevidentfromthecontext.

Toapplythefusionoperator,thefoursituationsmustbeweighedwiththenumberofpersonsthatendorsedthatsituation.Inotherwords,welookat

41󰀊S:=niSi

Ni=1

(2)

Fusionoptsforthesituationin{S1,S2,S3,S4}whichhasthelowestdistancetoS.Inotherwords,if“+Si”denotesadecisionforSiastheaggregatedcollectivejudgmentsetand“−Si”adecisionagainstSithen

+Si⇐⇒󰀘Si−S󰀘≤min󰀘Sj−S󰀘j=i

Ifwedefinedi:=󰀘Si−S󰀘,fusionranksthesituationSifirstifandonlyif

dij=i

Notethatnothinghingesonthechoiceofaparticularnormasadistancefunc-tionbecauseallnormsinfinite-dimensionalspacesare(ordinally)equivalent.Inparticular,theSiandSareallmembersofR3sothatthefusionprocedureisinvariantunderthechoiceofanorm:onlytheorderingofthedistancesmat-tersforthedecision.Inordertosimplifycalculationswerecommendtousethe1-norm(i.e.tosumtheabsolutevaluesofthethreecomponents)whichcorrespondstotheHammingdistance.

Theprinciplebehindthedistanceminimizationistheselectionofasituationthatisclosesttotheaverage.Table2illustratesthat,inthecourtexample,thesituationselectedbythefusionoperatorisS1={1,1,1}.ThisisbecauseS1is—amongthepossiblesituations—theclosesttothecollectiveaverage.Sinced1=2/7+2/7+4/7issmallerthananyotherdistancetoasituation,thefusionoperatorselectsS1.Wealsoseethat,withregardtoadecisiononS1,wecanapplyallthreeprocedures–PBP,CBPandfusion–toconjunctiveaggregationrules(A∧B↔C)aswellastodisjunctiverules(A∨B↔C)becausethelatterarerepresentableas((¬A∧¬B)↔¬C).Thejudgmentaggregationmechanismisabsolutelyisomorphic–accepting¬A∧¬BamountstoacceptingA∨Bandviceversa.

4

2.2Representationresults

Thereisalsoanintuitivelyunderstandablerepresentationofthemajorityfusionprocedure(henceforthFP).AssumethatavotersjudgeAtobetrue,bvotersjudgeBtobetrueandcvotersjudgeC↔(A∧B)tobetrue.(Ofcourse,a,b,andccanbecalculatedfromtheniandviceversa.)Thenseveralfactscanbeshown:2

Fact1Thefollowingclaimshold:1.min(a,b)≥c≥a+b−N

2.+S1⇔(a+c>N)∧(b+c>N).3.Ifd1Amongtheseclaims,wewouldliketodrawspecialattentiontothesecondone:

+S1⇔(a+c>N)∧(b+c>N)

(3)

Inordertosatisfyequation(3)andtoacceptS1intheaggregatedjudgment,asufficientnumberofpeoplehavetoendorsetheconclusionCindividually.Inparticular,atwo-thirdmajorityoneachofthepremisesissufficienttoguaranteethatfusionselectsthesituationS1.

From(3),wecanalsoderivethefollowingfact:

Fact2Let+Si(X)denoteadecisionforthesituationSiunderprocedureX.Then

•+S1(CBP)→+S1(FP)→+S1(PBP).•−S1(PBP)→−S1(FP)→−S1(CBP).

Hence,iftheCBPoptsforS1,sodoesfusion.AndiffusionoptsforS1,sodoesthePBP.However,iftheresultfromtheCBPisnegative,thenfusionismorecautiousthanthepremise-basedprocedure.Torecall,thePBPsuffersfromahighfalsepositiverate,i.e.itoftenendorsesC↔(A∧B)whenitisinfactfalse(cf.[7,8]).Fusionislessvulnerabletothismistake,asfact2shows.Wewillexpandonthispointinthesubsequentsection.

Itmightbesuspectedthatforanincreasingnumberofpremises,fusionmoreandmoreresemblesthepremise-basedprocedurebecausethecontributionofthepremisesoutweighsthecontributionfromtheconclusions.Nonethelessthisisonlytrueinthetrivialsensethatallthreeapproachesmakeitincreasinglyhardtoendorsetheconclusionevenwhenitistrue.

Fact3Leta1,...,amdenotethenumberofvotesforeachofthempremises.Then,

1.minai≥c≥

2All

󰀉m

i=1

ai−(m−1)N

proofsaregivenintheappendix.

5

2.+S1≡ai+c>N∀i∈{1,...,m}

Thisentailsthatforconstantpandlargem,c/N(i.e.thefractionofpeoplevotingforS1)isprobablysmall.ThisissobecausethelargenumberofpremiseswhichhavetobeaffirmedraisesthehurdlesforendorsingS1.Evenifthereisamajorityforeachofthepremises:thegreaterm,thehigherthesamplingvari-anceintheai,sothattheadditionalconditionai+c>Nbecomesincreasinglyhardtosatisfy.Hence,foralargenumberofpremises,FPwillresembleCBPandsetthestandardsforanendorsementofS1substantiallyhigherthanPBP.3AllthissuggeststhatfusionisintimatelyrelatedtoCBPandPBP.Indeed,wecanrepresentthetwolatterproceduresasdistance-basedprocedureswhen

t

weparametrizethesituationS1bymeansofS1:=(1,1,t)witht∈[0,∞].

t

(Foranyt,S1referstothesamerealworldsituationastheoriginalS1–bothpremisesandtheconclusionaretrue.Merelythedistancebetweenthissituationandthesubmittedsetofjudgmentisnowmeasureddifferently,inparticulart

S:=(a/N,b/N,tc/N).)Again,thesituationwhichminimizesthedistancetot

S,theaverageofthesubmittedjudgmentsets,ischosen.Thiselucidatestheconnectionbetweenfusion,CBPandPBP:

t

Proposition1LetS1=(1,1,t).Choosingthesituationwiththeminimum

tdistancetoSisequivalenttoPBPfort=0,yieldsFPfort=1andconvergestoCBPfort→∞.

Inotherwords,fort→0,thedistance-basedoperatorconvergesagainstthePBPwhichisattainedfort=0.4Ontheotherhand,whent→∞,thedistance-basedoperatorconvergesagainsttheCBP.Finally,fort=1,weobtaintheconventionalfusionoperator.5Fromnowon,weintendtheterm“distance-basedprocedure”torefertoallaggregationproceduresthatcorrespondtoaspecificvalueoft,includingt=∞.Thisgivesusacontinuumofdistance-basedapproaches,rangingfromthepremise-basedtotheconclusion-basedoperator,withfusionhavingamiddleposition.Wenowturntoanevaluationoftheprocedures.

3

3.1

Comparingthedistance-basedprocedures

Preliminaries

Inordertoinvestigatetheepistemicreliabilityofthefusionprocedure,weadoptaprobabilisticframework.Inparticular,weassigntoeveryvoteranindividualcompetencep∈(0,1)tomakeacorrectjudgmentaboutasinglepremise.Thismeansthatwhenapremise(eitherAorB)istrue,thevotergivesacorrectreportwithprobabilityp1=p,andequally,ifthepremiseisfalse,thevotergives

doesnotcontradicttheobviousfactthat,forfixedpriorprobabilities,theprobability

ofcorrectlydetectingS1approaches0asmgoestoinfinityforallthreeapproaches.

4WecanthinkofthatcaseasaprojectionofStontothehyperplanespannedbytheother

1

threesituations.

5Thevaluet=1isspecialbecauseitistheonlyvaluewheretheyes=1/no=0assignmentschemeintroducedintheprevioussubsectionappliestopremisesandconclusion.

3This

6

acorrectreportwithprobabilityp2=p.6Itwould,ofcourse,alsobepossibletoassigntwodifferentcompetencestothevoter,oneforcorrectlydiscerningAandoneforcorrectlydiscerning¬A.Butthatwouldcouplethecompetenceofthevotertothepriorprobabilitiesofthevarioussituations.Toseethis,notethat

p=p1P(A)+p2(1−P(A))

(4)

Wewouldoftenliketosaythatthereliabilityofthevotersisindependentofthepriorprobabilitiesoverthefoursituations.Theonlypossiblewaytoensurethisindependenceistoassumethattheprobabilityofafalsepositivereportonapremiseequalstheprobabilityofafalsenegativereport,inotherwords,p1=p2.7Then,theCondorcetJuryTheoremlinksthecompetenceofthevoterstothereliabilityofmajorityvoting:AssumethattheindividualvotesonapropositionAareindependentofeachother,conditionalonthetruthorfalsityofthatproposition.IfthechancethatanindividualvotercorrectlyjudgesthetruthorfalsityofAisgreaterthanfiftypercent(inotherwords,p>0.5),thenmajorityvotingeventuallyyieldstherightcollectivejudgmentonAwithincreasingsizeofthegroup.Therefore,theCondorcetJuryTheoremoffersanepistemicjustificationtomajorityvotingandmotivatestheuseofthePBPandCBPinthejudgmentaggregationproblem([2]).

Itshouldbenoted,though,thatanapplicationoftheCondorcetresultstojudgmentaggregationrequiresfurtherassumptionswhichwenowmakeexplicit.Theyarealsorequiredtoavoidcomputationalcomplexityandareformulatedasin[2]:

(i)ThepriorprobabilitiesthatAandBaretrueareequal(P(A)=P(B)).(ii)AandBare(logicallyandprobabilistically)independent.

(ii)Allvotershavethesame(independent)competencetoassessthetruthof

AandB(p).TheirjudgmentsonAandBareindependent.(iv)Eachindividualjudgmentsetislogicallyconsistent.Assumption(iv)entailsthatonlyfoursituationsarepossible:

tS1={A,B,C}=(1,1,t)tS3={¬A,B,¬C}=(0,1,0)

t

S2={A,¬B,¬C}=(1,0,0)tS4={¬A,¬B,¬C}=(0,0,0)

Moreover,assumption(i)andindependenceclaim(ii)entailthatwecanparametrize

thesetofpriordistributionsbyasingleparameterq:=P(A)=P(B).Fromtheindependenceassumptionswethenobtain

ascribeanindividualcompetenceonlyforvotingonpremises,notforvotingonany

proposition(suchasA∧B).Indeed,itfollowsthatgivenanindividualvotingcompetenceponAandB,thevotingcompetenceonA∧Bisp2=p([8]).However,inmanycontextsitisreasonabletoassignindividualvotingcompetencetoonlyacertainkindofpropositions.E.g.inalegalcasethiswouldbepropositionsas“PhadcontractualobligationnottodoX”or“PactuallydidX”,butnotonpropositionsas“Pshouldgotojail”.

7Settingp=palsoanswersList’sconcerns([8])thatforaverylowvalueofporp,the1212

votersarebadattrackingthetruesituationalthoughtheoverallreliabilityp,asdefinedin(4),canstillbehigh.Regardlessofwhetherthispointisreallyconvincing,settingp1=p2killstwobirdswithonestone:wecircumventList’sobjectionandwedecoupleoverallreliabilityandpriorprobabilities.

6We

7

ttttP(S1)=q2;P(S2)=P(S3)=q(1−q);P(S4)=(1−q)2

Theprobabilitythatadistance-basedprocedurechoosestherightconclusion

canbecalculatedvia

P(G)

:=P(Adistance-basedprocedureselectstherightconclusion)=

tttP(S1)P(+S1|S1)

4󰀊i=2

tt

where“+S1”denotesacollectivejudgmentthatselectsthesituationS1andthe

t

P(Si)-termscanbereplacedbythecorrespondingq-terms.

+

tttP(Si)P(−S1|Si)

3.2Resultsandgeneralizations

Withtheaboveequationsinhand,wecannowcomparethefusionprocedure

(FP)tothePBPandtheCBP.BovensandRabinowicz([2])showthatthePBPisalwaysbetteratidentifyingthecorrectsituation,whiletheCBPissometimesbetteratselectingtherightconclusion.ThismeanseithertoacceptortorejecttS1asthecorrectsituation,and,incaseofarejection,tobesilentonwhethertttS2,S3orS4istrue.Indeed,inavarietyofrealaggregationproblems,itis

t

mosturgenttocometoaverdictwithregardtoS1anditislessimportant

ttt

todiscernbetweenS2,S3andS4(e.g.becausetheyhavethesamepracticalconsequences).However,thatdoesnotmeanthattheaggregationprocedures

t

neglectthereasonsforeitheracceptingorrejectingS1:Thenumberofvotesforeachpremiseplaysasubstantialpartinalldistance-basedapproachestojudgmentaggregation,withtheobviousexceptionofCBP.Thecomplementaryproblemofsituationselectioniscoveredindetailinasequelpaper([4]).

10.90.80.70.6010.90.80.70.60102030401020304010.90.80.70.601020304010.90.80.70.6010203040Figure1:ReliabilityofPBP(triangles),FP(stars)andCBP(diamonds)asafunctionofN,forvariousvaluesofpandafixedvalueofq=0.3.Upperleftfigure:p=0.56.Upperrightfigure:p=0.64.Lowerleftfigure:p=0.72.Lowerrightfigurep=0.8.

Figures1-3depictthereliabilityofPBP,CBPandFPforvariousvaluesofp,qandoddvaluesofN.Firstwewouldliketodiscussfigure1.Itturnsoutthat

8

10.90.80.70.6010.90.80.70.60102030401020304010.90.80.70.6010.90.80.70.601020304010203040Figure2:ReliabilityofPBP(triangles),FP(stars)andCBP(diamonds)asafunctionofN,forvariousvaluesofpandafixedvalueofq=0.5.Upperleftfigure:p=0.56.Upperrightfigure:p=0.64.Lowerleftfigure:p=0.72.Lowerrightfigurep=0.8.

10.90.80.70.6010.90.80.70.60102030401020304010.90.80.70.6010.90.80.70.601020304010203040Figure3:ReliabilityofPBP(triangles),FP(stars)andCBP(diamonds)asafunctionofN,forvariousvaluesofpandafixedvalueofq=0.7.Upperleftfigure:p=0.56.Upperrightfigure:p=0.64.Lowerleftfigure:p=0.72.Lowerrightfigurep=0.8.

9

forrelativelysmallvaluesofp(p=0.56,0.64),thepremise-basedprocedure

t

toooftenerroneouslyendorsesS1,andespeciallysoforsmallvaluesofN.Inthiscase,amajorityforapremisecanemergebymererandomsamplingeffectsalthoughthepremiseisactuallynotsatisfied.ThereforePBPisinferiortobothFPandCBPinsuchcircumstances.Forhighervaluesofp,however,thethreeproceduresnearlycoincideanddonotdiffermuch.Thisisespeciallysalientforp=0.8.Figure2confirmsthelocalfailureofPBPforamodestp(p=0.56).However,wealsoseethatforintermediatevaluesofp(p=0.64),PBPclearlydominatesthetwootherapproacheswhereasthereisagainnosignificantdifferencebetweenPBPandtherestforhighvaluesofp.

t

ThesuperiorityofPBPismostpronouncedinFigure3whereq=0.7,i.e.S1isthemostprobablesituation.Foranyvalueofpsmallerthan0.8,PBPclearlyoutperformsthetwootherprocedures.Thatisnotsurprising:thegreaterq,the

tt

moreimportantisittoavoiderroneousrejectionofS1,justbecauseS1occursmoreoften.Fact2hasestablishedthat,amongthethreescrutinizedprocedures,

t

PBPismostinclinedtowardsacceptingS1,asalreadynotedby([2],[8]).This

t

“optimism”towardsS1naturallypaysoffintermsofoverallreliabilitywhen

8

qisquitelarge.Ontheotherhand,weseethatCBPfailstobenefitfromthegreaterstabilityinthedatawhichaccompaniestheincreasingnumberofvoters.Especially,weseethatCBPperformsquitepoorlyforlargevaluesofNincomparisontotheotherprocedures.Besidesweseeagainthatallthreeproceduresarealmostequallyreliableforp=0.8becausethehighindividualreliabilityguaranteesthatanyprocedureiswellprotectedagainsterror.

TheconcreteobservationsforlargeNintheaboveexamplescanbegeneral-ized.Weperformanasymptoticanalysisofthedistance-basedproceduresinageneralframework,buildingontheparametrizationalreadyusedinproposition

t

1.ConsiderfirstthecasethatS1istrue.

tProposition2AssumethatS1=(1,1,t)isthetruesituation.ThenP-almostsurely(P-a.s.)forN→∞:

√

2t2+2t+1−1ttt

+S1⇐⇒d1pt:=

j=12t

√

Inparticular,thistranslatesasp>0.5forPBP,p>(5−1)/2forFPand√

p>1/2forCBP.

Thefollowingcorollaryassertsthatp>ptisbothnecessaryandsufficientinordertoensuretheP-a.s.correctconclusionselectionforincreasinggroupsize:Corollary1Forthegroupsizegoingtoinfinity(N→∞),thedistance-basedproceduresselecttherightconclusionP-a.s.ifandonlyifp>pt.

Putanotherway,P(G)→1ifandonlyifpislargerthanthespecifiedthreshold.Hence,PBPhasbetterasymptoticpropertiesthanfusionbecauseforalargenumberofvoters,iteventuallybecomesperfectlyreliableforp>0.5√whereasfusion√requiresthestrongerp>(5−1)/2.CBPrequirestheevenhigherp>1/2.ThissuperiorityofPBPforlargevotinggroupsisexemplifiedinallthreefigures.Moreover,theasymptoticresultsexplainwhyCBPisnot

evenconjecturethatthereisathresholdforq(dependentonp)sothatforanyN,

PBPismorereliablethananyotherdistance-basedprocedure(t∈(0,∞)).Wewouldliketoprovesucharesultinfuturework.

8We

N→∞

10

monotonouslyincreasingasafunctionofNforp=0.56orp=0.64.ThesameholdsforFPwithregardtop=0.56.However,thefiguresalsoteachusthatforlargevaluesofp,theasymptoticresultscarrylittleimportancefortheactualreliabilitybecauseallthreeprocedurestendtoagreequickly.Furthermore,theasymptoticpropertiesarenotalwayscorrelatedwiththeperformanceinsmallvotinggroups:Forsmalltomoderatevaluesofp,qandN,FPandCBPoutperformPBP–seetheuppergraphsinfigure1and2.Asalreadymentioned,webelievethatthisisduetorandomsamplingeffectswhichoccurinsmallvotinggroups.

Wecansummarizetheresultsasfollows:Forhighvaluesofp(approximatelyp>0.75),allthreeexaminedproceduresareveryreasonable.Choosinganaggregationmethodamongtheinfinityofdistance-basedproceduresdoesnotmakemuchofadifference.Onlyformoderatevaluesofp(p∈[0.5,0.75]),thereisarealdifferencebetweentheaggregationprocedures.Itturnsoutthatthe

t

priorprobabilityofS1,q2,playsacrucialrolehere.Roughly,wecansaythatthehigherqandthehigherN,themoreshouldwebeinclinedtowardsPBP,whereasforsmallgroupsandmodestq,FPorevenCBPcanbethebetterchoice.IncomparisontoCBP,FPhasthevirtueofnotperformingtoobadlyforlargesamplesandmediumvaluesofp.Forpotentialapplications,itmightbeinterestingtonotethatinalotofjuryandpaneldecisions,thenumberofvotersisquitesmall,typicallyN∈{5,7,...,15}.Hence,especiallywhenwehavesomereasonsnottofullytrustthevoterscompetence(take,forinstance,alaymenjuryinacriminaltrial),wehavearationaleforapplyingthefusionoperator.Forsuchcases,wealsosuggestfurthercalibrationsoftinordertocombinethepowerofPBPwiththeconservativenessofFP,e.g.t=0.5.Bycontrast,whenwefacealargenumberofvoters,forinstanceinaplebiscite,recommendingPBPisthesafestoptionduetotheasymptoticsuperiority.Suchcalibrationscanbefurtherrefinedbyconsideringtherelativeseverityofadecisionerror.

t

E.g.iferroneousacceptanceofS1wereinaspecificsituationmuchworsethan

t

erroneousrejectionofS1,wewouldtendtosetttoahighervaluethaniftheoppositeweretrue.

4

4.1

TheBayesianApproach

GeneralRemarks

TheprobabilisticframeworkwhichweusedfortheevaluationofPBP,CBPandFPcanbetransferredtoafullBayesianapproach,too.InaBayesianapproach,wehaveapriorprobabilitydistributionoverthesituationsS1toS4,givenagainby(q2,q(1−q),q(1−q),(1−q)2).Wetreatthejudgmentsofthevoters(callthemV)asincomingevidencewhichweusetoupdatethepriorprobabilitiestoaposteriordistributionoverS1toS4:

P(Si|V)

=

P(Si)P(Si|V)

P(V)

Thisposteriordistributiondescribesourrationaldegreeofbeliefinthevarioussituations,giventheverdictsofthevotersandtheirindividualreliability.Thenwebaseourdecisionexclusivelyonthatposteriordistributionandtheutilitymatrixwhichdescribestheactualdecisionproblem.

11

acceptS1(“+S1”)rejectS1(“−S1”)S1istrue

10S1isfalse

01Table3:TheutilitymatrixthatcorrespondstousingP(G)asabenchmarkfortheperformanceoftheaggregationprocedures,shownasafunctionofthepossibleactionsandstatesoftheworld.

Wearenowinterestedintheaverageprobabilitythattherightconclusion(S1or¬S1)isselected.Inotherwords,wewanttocalculateP(G)anduseitasathebenchmarkforthevariousaggregationprocedures.KeepinginmindthatP(G)=P(S1)P(+S1|S1)+P(¬S1)P(−S1|¬S1),thiscorrespondstoadecisionproblemwhereutility1isassignedtoacorrectconclusionselectionand0toawrongconclusionselection(seetable3).9

TheConditionalBayesPrinciple([1],p.8)tellsusthat,relativetoagiven(posterior)distribution,weoughttotaketheactionthatmaximizestheexpectedutility.Facedwiththeaboveutilitymatrix,wewilloptforS1ifandonlyifP(S1|V)>0.5.

Wecangeneralizethatprincipletothefollowingdecisionrule:foranysetofjudgmentsthatwillbeobserved,taketheactionthatmaximizesexpectedutilityrelativetotheposteriordistributionwhichweobtainbyupdatingonthevoters’judgments.Indeed,itcanbeshownthatsuchadecisionruleisaBayesrule,i.e.adecisionrulethatminimizestheexpectedriskwithregardtothepriordistributionamongalldecisionrules.10HenceweseethatthedecisionrulegivenbytheBayesianapproachisoptimalintherisk-minimizing(orutility-maximizing)sense.Wecanbasealldecisionssolelyontheposteriordistributionandtheproblem-specificutilitymatrix.

Thishasanumberofimplications.First,alldatathatdonotaffecttheposteriordistributionoftheSicanbeneglected.TheposteriordistributionisuniquelydeterminedbythenumberofvotesforAandB,inotherwords,thestatisticsaandb.Allfurtherinformationisirrelevant,giventhevaluesofthosefunctions.Technicallyspoken,aandbaresufficientstatistics.11Onceweknowthevaluesofaandb,wecantotallyneglecthowmanypeopleendorsedtheconclusion.Thisvindicatesanintuitionunderlyingthepremise-basedprocedure:completeinformationaboutthepremisesisallweneedtomakeareliabledecision.Indeed,theRao-BlackwellTheorem([1],p.41)guaranteesthatthereisnoinformationthatcanimprovethedecisionrulebeyondwhatiscontainedinthesufficientstatistics.Moreprecisely,anydecisionrulecanbeimprovedinawaythatitisonlyafunctionofthesufficientstatistics.

Thisalsoimpliesthatboththepremise-andtheconclusion-basedapproachcannotbeoptimal:Botharebasedon0-1statisticsthatmeasurewhethertherearemajoritiesforA,BorC.Butthosestatisticsareneithersufficientnorjointlysufficient.Toomuchinformationgetslostwhenonlycheckingthemajorities.Asimilarresultholdsforfusion,wherethedecisionontherightconclusionisonlybasedonthestatistics(a+c)and(b+c)whichareneither(jointly)sufficient.Hence,noneofthethesethreeapproachescanbeoptimal.Contrarily,bythe

otherwords,erroneouslyoptingforS1isequallydevastatingaserroneouslyoptingfor

¬S1.Asimilarmatrixcanbefoundforthesituationselectionproblem,seeagain[4].10Cf.Result1in[1],p.159.

11AstatisticTissufficientwithregardtoanunknownparameterΘwhenP(Θ=ϑ|T=t)=P(Θ=ϑ|X=x)whereXdenotesthefulldata.

9In

12

10.80.60.40.2

00.20.40.60.81Figure4:TheprobabilitythattheBayesianprocedureidentifiestherightcon-clusionasafunctionofthecompetenceofthevoterspforq=.7andN=11(dottedline),N=21(dashedline)andN=51(fullline).

aforementionedtheorem,theBayesianapproachprovidesanupperboundforthereliabilityofalldecisionrules.WenowturntotheperformanceoftheBayesianapproachandcompareitsreliabilitytotheproceduresdiscussedintheprevioussection.

4.2ResultsandDiscussion

InFigure4.2,wehaveplottedthereliabilityoftheBayesianaggregationpro-cedureasafunctionoftheindividualreliabilityforq=0.7andvariousgroupsizes(N=11,21,51).WeseethattheBayesianprocedureisalmostperfectlyreliablewhenpisfarfrom0.5.FromtheaboveargumentsitisclearthattheBayesianreliabilityconstitutesanupperboundforallotherprocedures.Also,weseethatthereliabilityismonotonouslyincreasinginN,anditapproaches1(pointwise)forallvaluesofpexceptforasmallneighbourhoodof0.5.Itcanevenbeshownthatforallvaluesofpexceptforp=0.5,theBayesianprocedureeventuallyselectstherightconclusion.

Proposition3Foranyp=0.5,theBayesiandecisionruleeventuallyselectstherightconclusionP-a.s.whenN→∞.

ThisdrawsourattentiontoanotherfeatureoftheBayesiananalysis:Itissymmetricasafunctionofp.SoBayesianaggregationisperfectlyreliableevenwhentheindividualvotersareveryunreliable.ThisshedslightonasubstantialpremiseoftheBayesianapproach,namelythatknowledgeofpandqisrequiredtoupdatethepriordistribution.Inotherwords,bothpandqhavetobetransparenttothedecisionmaker.Then,itisnomoresurprisingthattheBayesianprocedureishighlyreliableforp≈0:Whentheaggregatorsknowthatthevotersnearlyalwayssubmitwrongjudgments,theywilljustreplacetheindividualjudgmentsonthepremisesbytheirnegations.So,whenahighlyunreliablevotersubmits(1,0,0),thisamountstothesubmissionof(0,1,0)byahighlyreliablevoter.ThereforetheBayesianprocedureworksperfectlyfineforlowvaluesofp.

Ontheotherhand,itisquestionablewhetherknowledgeofpandqcanreallybepresupposed.Suchtransparencyishardlyrealisticandnotapplicableinawideclassofcases.Forexample,anykindofassigningpriorprobabilitiesis

13

frowneduponinlegaldecisionmaking.Evenmoredisturbing,ourestimateoftheindividualvotingcompetencemaybegrosslymistaken.Forexample,as-sumethatweerroneouslyclaimthatp=0.5,i.e.webelievethevoterstoberandomizers.FortheBayesian,thismeansthattheresultsofthevotingprocesshavenoimpactontheposteriordistribution:itequalsthepriordistribution.Evenifthevotersunanimouslyendorsebothpremisesandtheconclusion,thishasabsolutelynoimpact.Inparticular,whenP(S1)≤0.5,evenanunanimousendorsementofS1doesnotleadtotheacceptanceofS1.Thisisaresultwhichweintuitivelyfindabsurd:NotonlydoestheBayesianrecommendationconflictwiththeprincipleofunanimity,thedataalsosuggesttoreviseourestimateofpbecauseunanimityismuchlesslikelyforrandomizingvotersthanforcompe-tentvoters.Forsuchanextremesetofsubmittedjudgments,wewouldrathertendtoacceptS1asitisrecommendedbyalldiscusseddistance-basedproce-dures.Thedataseemtofalsifyourpreviousestimatep=0.5.Sucharevisionis,however,notpossibleinagenuineBayesianframeworkbecausethiswouldamounttodouble-countingthedata:onceforelicitinganestimateofpandaf-terwardsforupdatingthepriordistribution.Assigningapriordistributionoverpandaveragingtheresultsoverthepriordistributionofp(“Bayesianmodelaveraging”)wouldmitigate,butnoteliminatetheeffect.WeleaveitopentofutureresearchwhethertheBayesianaggregationprocedurecanbeprotectedagainstsevereestimationerrors.Inanycase,theBayesianapproachcarriesaconsiderableestimationriskwhenpishardtoelicit.

Inseveralcontextsitis,ofcourse,notawkwardtoassumethatpismoreorlesstransparent,e.g.whenwehaveNindependentmeasuringinstrumentsinascientificexperiment.InsuchacaseweshouldalwaysuseBayesianupdatingandmakeadecisiononthebasisoftheposteriordistribution.Butinavarietyofcaseswherehumanjudgmentsareaggregated,e.g.whenajuryhastodecideupontheliabilityofthedefendantortoawardordenytenuretoafacultymember,thecompetenceofthejurymaybehardtoestimate.Inparticular,theremaybenorelativesuccessfrequencyasabasisforanestimateofthevotingcompetence.Moreover,theBayesianprocedurewillsometimes,namelyforlowvaluesofp,recommendtodoexactlytheoppositeofwhatthevotersarethinking.ThispreventstheBayesianprocedurefrombeingappliedinmanypracticalcases.Itfaresbestwhenthedecision-makerhasasensibleestimateofpandwhenhedoesnotneedtheconsentofthevoterstotakeanaction.12Tosummarize:TheBayesianapproachhastheadvantagethatthedecisionprocedurecanbeflexiblyadaptedtotheparticularproblembychangingtheutilitymatrix.VariousjudgmentaggregationproblemsarecharacterizedbydifferentutilitymatricestowhichtheBayesianprocedurecanbeflexiblytrans-ferred.E.g.sendingsomeonetojailerroneouslyhasamuchhigherassociatedlossthansettingfreeaguiltyperson.Butdenyingtenuretoanoutstandingresearcherandteachermightbeworsethangivingtenuretoapersonwhoisacapableresearcherbutonlyamediocreteacher.Inotherwords,theutilitiestelluswhichkindoferrorwehavetoavoid.TheBayesianapproachnaturallyincorporatesthevariationinthelossesassociatedwithawrongdecisionofacertainkindwhereasdistance-basedapproachesasCBP,PBPandFParenotsensitivetotheseverityofaparticularkindoferror.TheBayesianframeworkcanbenaturallyextendedtoconnectionrulesdifferentfromtheconjunctionofthepremises,too.Ontheotherhand,thedistance-basedproceduresdonot

far,wehavebeensilentonthepossibilityofstrategicvotingwhichcanoccurinthe

Bayesianmodelaswellasinthedistance-basedapproaches.Thereforeweassumethatthevotershavenointerestinaspecificconclusion–otherwisenovoterwouldsubmitthejudgmentsetS2orS3.

12So

14

involveacomplicatedcalculationofposteriorprobabilitiesandtheyareeasiertoapplytorealdecisionproblems.

5Conclusions

Bymeansofthet-parametrizationinsection2,wehaveintegratedPBP,CBPandFPintoacontinuumofdistance-basedprocedures,withFPtakingamiddlepositionbetweenthetwoextremesPBPandCBP.Therebyweachieveaconcep-tualunificationwithregardtothetraditionalmethodsofjudgmentaggregation.Withregardtoselectingtherightconclusion,thedistance-basedproceduresaretheoreticallyinferiortotheBayesiandecisionprocedurewhichalsoenjoyshighflexibility.Nevertheless,theBayesianapproachcanonlybeappliedwhentheindividualcompetencepandthepriorprobabilityqofeachpremisecanbeelicited.Often,estimatingpmightbeassociatedwithaunpredictableestima-tionrisk,deterioratingtheperformanceoftheBayesianapproach.Thiswasillustratedinthetoyexampleoftheprevioussection.Inpractice,workingwithanestimateofpmightalsobeprohibitedbyexternal,pragmaticconstraints.Themoreweareuncertainaboutp,themoreweshouldbeinclinedtoapplyadistance-basedapproachwhichperformsreasonablywellforvariousvaluesofp.Themostsuitablevalueoftthendependsontheactualnumberofvotersandtheestimatedpriorprobabilities.Distance-basedapproaches,althoughnotstrictlyoptimalfromatheoreticalpointofview,mightbeareasonablecompro-misebetweentwoends:toneglectanestimateofindividualcompetenceintheactualdecision-makingandtohaveaprocedurethatreliablyselectstherightconclusion.Amongtheseprocedures,thosewhichresemblethepremise-basedprocedure(t󰀇1)willusuallyperformbest.Theprecisecalibrationoftinanactualapplicationissensitivetothespecificsocialdecisionproblemandwillbeinvestigatedinfurtherwork.Moreover,oursequelpaper[4]examinestheperformanceofthediscussedproceduresattrackingtherightsituation.

A

A.1

Calculationaldetails

Propertiesofthefusionoperator

WeexaminethecaseA∧B↔C.ThereareNvotersandweassumethatNisanoddnumber≥3.n1votersvoteforS1=(1,1,1),n2forsituationS2=(1,0,0),n3forsituationS3=(0,1,0)andn4forsituationS4=(0,0,0).Obviously,N=n1+n2+n3+n4.Fusionchoosesthemodelwhichhasthe󰀉4

1

lowestdistancetotheaverageofsubmittedjudgmentsets,S:=Ni=1niSi(cf.equation(2)).Forreasonsofconvenience,weworkwiththeHammingdistancewhichcorrespondstothe1-NorminrealEuclideanvectorspaces.(Asmentionedinthemaintext,allnormsareordinallyequivalent).LetadenotethenumberofvotersthatvoteforpremiseAandbthenumberofvotersthatvoteforB.Accordingly,letcdenotethenumberofvotersthatvotefortheconclusionC.Letdi:=󰀘Si−S󰀘1.Hence,fusionranksmodelSifirstifandonlyifdi15

ProofofFact1:Thefirstclaimistrivial.Fortherest,notethatN(d2−d1)N(d3−d1)N(d4−d1)

==

(N−a)+b+c−(N−a)−(N−b)−(N−c)=2b+2c−2Na+(N−b)+c−(N−a)−(N−b)−(N−c)=2a+2c−2N

=a+b+c−(N−a)−(N−b)−(N−c)=2a+2b+2c−3N

Thefirsttwoequationsyieldd1Nandd1N.Ifthesetwoconditionsaresatisfied,itfollowsthat

N(d4−d1)>a+b−N>0

(5)

becausea+c>Nalsoimpliesa>N/2andanalogouslyforb.Thus,alsod1ProofofFact2:Weonlyprovethefirstclaimofthefact,thesecondfol-lowsbycontraposition.Apositivereportintheconclusion-basedprocedure(“+S1(CBP)”)occursifandonlyifc>N/2.Sincea,b≥c,thisalsoimpliesa+c>Nandb+c>N.SofusionoptsforS1,too,invirtueoffact1.Thetwolatterinequalitiesimplythata,b>N/2(again,becausea,b≥c).Hence,iffusionoptsforS1,sodoesthepremise-basedprocedure.󰀁ProofofProposition1:

1

(N−a+N−b+t(N−c))N1dt(a+N−b+tc)3=Ndt1=

1

(N−a+b+tc)N1dt(a+b+tc)4=Ndt2=

tt

AdecisionforS1ismadeifandonlyifdt1t

wealsogetc≤N/2andt(N−c)≥tcforallvaluesoft.Henceeitherdt1≥d2

tt

ordt1≥d3sothatS1isrejectedindependentofthevalueoft.

t

Thereforeassumethata,b>N/2.ThenadecisionfortheconclusionS1

t

amountstodt12a−N+t(2c−N)>02b−N+t(2c−N)>0

2a−N+2b−N+t(2c−N)>0

Obviously,thefirsttwoinequalitiesentailthethirdone.

Fort=1,thisleadstotheusualconditionsa+c>Nandb+c>N,asshowninfact1.

Fort=0,theinequalitiesaresatisfiedifandonlyifa>N/2andb>N/2,i.e.whenthereisamajorityforeachofthepremises.HenceweobtainPBP.

Finally,forverylarget,the2a−Nand2b−Ntermsdropoutofthepicture.

t

HenceweacceptS1ifandonlyifc≥N/2.CBP–accepttheconclusionifandonlyifitisendorsedbyagenuinemajority–isobtainedinthelimitt→∞.󰀁Remark:Itisasimplefactoflinearalgebrathattherelativeorderingbetween

tt

thedistancesdt2,d3andd4isnotaffectedbythevalueoft.So,theproofwouldgothroughevenifweaimedatchoosingtherightsituationinsteadoftherightconclusion.

ProofofFact3:Thefirstinequalityistrivial.For󰀉thesecondone,the

m

maximal󰀉setofvoterswho󰀉donotaccepttheconclusionisi=1(N−ai).Hence,

mm

c≥N−i=1(N−ai)=i=1ai−(m−1)N.

16

Forthesecondclaim,firstassumethatthereisanisothatai+c≤N≡N−c≥ai.Then,bymeasuringdistancebythesupremumnorm,weseethat

sup(N−a1,N−a2,...,N−am,N−c)=≥

N−c

sup(N−a1,N−a2,...,N−ai−1,ai,N−ai+1,...,c)

sothatSi:=(1,1,...,1,0,1,...,0)isclosertoSthanS1=(1,1,...,1).HenceS1isrejected.

Nowassumethat∀i:ai+c≥N.ThenS1ischosenifandonlyifN−cA.2

Asymptoticalbehaviourofthedistance-basedproce-dures

t

ProofofProposition2:UnderS1,aisBN,p-distributed.Similarly,b∼BN,pandc∼BN,p2.AllthreevariablesaresumsofNindependentandidenticallydistributedrandomvariablessothattheStrongLawsofLargeNumbersap-plies.ItfollowsthatP-a.s.a/N,b/N→p,c/N→p2.Inparticular,withtheexceptionofasetofmeasurezero,

abc

∈(p2−ε,p2+ε),,∈(p−ε,p+ε)(6)NNN󰀂2󰀂󰀂Choose(3+3t)ε:=2tp+2p−(1+t)󰀂.Asimplecomputationyieldsthat

√

2tp2+2p−(1+t)>0ifandonlyifp>pt:=(1/2t)(2t2+2t+1−1)for

tt

t=0andp>0.5fort=0.RecallthatS1ischosen(“+S1”)ifandonlyif

󰀃c󰀄a

2−1+t2−1>0NN

(7)󰀃󰀄bc

2−1+t2−1>0NN

∀ε≥0∃N0∀N≥N0:

(cf.theproofofproposition1).Andindeed,forp>ptandN≥N0,

󰀃c󰀄a

2+t2−1NN

>2p−2ε−1+t(2p2−2ε−1)

=

(3+3t)ε−2ε−2tε>0

Exactlythesamecomputationcanbedonefortheotherequationin(7).Thus,

t

ifp>pt,P-almostsurelytherightconclusioniseventuallychosenifS1istrue.Assumeontheotherhandthatp≤pt.Then,invirtueof(6),forN≤N0,

󰀃c󰀄a

2+t2−1NN

<2p+2ε−1+t(2p2+2ε−1)

=<0

t

andeventuallyselectsthewrongconclusionisP-a.s.selectedifS1istrue.√Fort→∞(CBP),asimpleasymptoticanalysisyieldstheconditionp>1/2,(−3−3t)ε+2ε+2tε

17

whereast=0(PBP)amountstop>0.5.Finally,inthecaset=1(FP),we√

getthethresholdp>(5−1)/2.󰀁

t

ProofofCorollary1:Proposition2hasalreadyprovedthat,ifS1istrue,therightconclusioniseventuallychosenP-a.s.ifandonlyifp>pt.Threeremainingcasesaretoexamine.

t

(a)S2istrue.Ifp>0.5wewillP-a.s.getbt

,independentofthevalueoft.rejectS1t

(b)S3istrue.Thesameas(a)duetosymmetry.

t

(c)S4istrue.Ifp>0.5wewillP-a.s.geta,bS1isrejected,independentofthevalueoft.

Sincep>ptentailsp>0.5,alldistance-basedprocedureseventuallychoosetherightconclusionP-a.s.ifandonlyifp>pt.󰀁

A.3Bayesianposteriors

WecalculatetheposteriorprobabilityofS1conditionalontheobservedvariablesaandb,thenumberofvotesforpremiseAandB,respectively.Calculationsarecompletelyanalogousfortherivallingmodels.LetLij(a,b)bethelikelihoodratioofsituationSitosituationSjasafunctionofthedatax.Then,wehave

󰀅󰀅󰀅

󰀆2b−N󰀆2a+2b−2N󰀆2a−N

󰀅󰀅󰀅

󰀆2a−N󰀆2a−2b󰀆2b−N

L21(a,b)=L41(a,b)=L42(a,b)=

1−pp1−pp1−pp

L31(a,b)=L32(a,b)=L43(a,b)=

1−pp1−pp1−pp

1

NotethatLij(a,b)=L−ji(a,b).WecannowcalculatetheposteriordistributionifavotesforpremiseAandbvotesforpremiseBaresubmitted:

P(S1|a,b)

=

P(S1)P(a,b|S1)P(S1)P(a,b|S1)

=󰀉4P(a,b)i=1P(Si)P(a,b|Si)󰀇

4󰀊P(Si)1+Li1(a,b)

P(S)1i=2

󰀈−1

=

󰀇=

1+

1−q1−qL21+L31+qq

󰀅

1−q

q

󰀆2

󰀈−1L41

=:

M−1

Inotherwords,wedenotetheterminsidethesquarebrackets(withoutthe

18

exponent)byM.ForanyeventE,duetoindependence,

P(E|S1)

=

󰀆󰀅󰀆N󰀊N󰀅󰀊NNaN−ap(1−p)pb(1−p)N−b1E

ab󰀆󰀅󰀆N󰀊N󰀅󰀊NNaN−a

p(1−p)(1−p)bpN−b1E

ab󰀆󰀅󰀆N󰀊N󰀅󰀊NN

(1−p)apN−apb(1−p)N−b1E

ab󰀆󰀅󰀆N󰀊N󰀅󰀊NN

(1−p)apN−a(1−p)bpN−b1E

ab

a=1b=1

P(E|S2)

=

a=1b=1

P(E|S1)

=

a=1b=1

P(E|S4)

=

a=1b=1

1Edenotesthe0-1indicatorfunctionoftheeventE.AssumenowthatS1is

true.WhenwewanttocalculatetheprobabilitythattheBayesiandecisionproceduregetstheconclusionright,wecomputetheprobabilitythatMissmallerthan2,giventhepriordistribution:

P(+S1|S1)

=

P(M<2|S1)

whichcanbecalculatednumericallyaccordingtotheprobabilitydensitiesgivenabove.Analogously,

P(−S1|¬S1)

=P(−S1|S2∨S3∨S4)

󰀁1󰀋2

q(1−q)P(−S|S)+q(1−q)P(−S|S)+(1−q)P(−S|S)=121314

1−q2󰀁1󰀋2

=2q(1−q)P(M≥2|S)+(1−q)P(M≥2|S)24

1−q2sothatwecancomputeP(G).

A.4Bayesianasymptotics

ProofofProposition3:Itisclearthatforp=0.5,thereliabilityoftheBayesiandecisionruleisconstantinN,becausethejudgmentsofthevotersdonotaffecttheposteriordistribution.Twocasesremaintoexamine.

(a)p<0.5.AssumeS1istrue.Then(1−p)/p>1andinthelongrun,almostcertainlya,b(b)p>0.5.Theproofissimilar:AssumeS1istrue.Then(1−p)/p<1andinthelongrun,almostcertainlya,b>N/2whichentailsL21(a,b)=((1−p)/p)2b−N→0,etc.󰀁

References

[1]J.O.Berger.StatisticalDecisionTheoryandBayesianAnalysis.Second

Edition.Springer,NewYork,1985.

19

[2]L.BovensandW.Rabinowicz.Democraticanswerstocomplexquestions.

Anepistemicperspective,Synthese,150:131–153,2006.[3]F.DietrichandC.List.Arrow’stheoreminjudgmentaggregation.Social

ChoiceandWelfare,2006.[4]S.Hartmann,GPigozzi,andJ.Sprenger.Judgmentaggregationandthe

problemoftruth-tracking.Inpreparation.[5]S.KoniecznyandE.Gr´egoire.Logic-basedapproachestoinformationfusion.

InformationFusion,7:4–18,2006.[6]S.KoniecznyandR.Pino-P´erez.Mergingwithintegrityconstraints.InFifth

EuropeanConferenceonSymbolicandQuantitativeApproachestoReason-ingwithUncertainty(ECSQARU’99),pages233-244,1999.[7]C.List.Theprobabilityofinconsistenciesincomplexcollectivedecisions.

SocialChoiceandWelfare,24(1):3-32,2005.[8]C.List.Thediscursivedilemmaandpublicreason.Ethics,116(2):362–402,

2006.[9]C.List.Judgmentaggregation-Abibliographyonthediscursivedilemma,

thedoctrinalparadoxanddecisionsonmultiplepropositions.2007http://personal.lse.ac.uk/LIST/doctrinalparadox.htm[10]C.ListandP.Pettit.Aggregatingsetsofjudgments:Animpossibility

result.EconomicsandPhilosophy,18:89-110,2002.[11]G.Pigozzi.Beliefmergingandthediscursivedilemma:anargument-based

accounttoparadoxesofjudgmentaggregation.Synthese,152(2):285-298,2006.

20