hexis-mod-tr

Model type: bert

Downstream task: binary text classification

Finetuning Emissions: 0.21332609 CO₂-equivalents [CO₂eq], in kg


Benchmarks

Accuracy: 0.932139794168096
F1: 0.9397200266641272

Accuracy: 0.9450180169237621
F1: 0.8518437704560332

  • Self-consistency
Accuracy: 0.9999826444631499
F1: 0.9999807162003201


Notes
  • Self-consistency test: Evaluation on all training data.
  • Train-test splits: If the dataset is not divided into train and test portions, a 70-30 train-test split is performed.

Debiasing

Test terms from StereoSet as contained in the training data. Showing the difference in Attention Entropy before and after Optimization of Information Flow. Unit: Entropy Bits

Hindu

0.0012172932028069062

Nepal

0.0008695442827623223

Türkçe

0.0006113826890316312

Muhammed

0.0005923082495714869

Afrika

0.00046043195288976294

Suriye

0.0003992863877527094

Venezuela

0.0003756101094843576

Katar

0.0003670109769412175

akademik

0.0002366481275872143

pilot

0.00023389640517340948

Avrupa

0.00022967501283381684

hakem

0.00021394641767332454

ebe

0.00019399643017323963

Irak

0.00015501369597681713

Yunan

0.00014455141805018432

İsveç

9.699821508661982e-05

Hindistan

7.223271336237646e-05


6.283099511435575e-05

Çin

5.5034448276096346e-05

Kadın

4.884307284503551e-05

Rusya

3.47717650468205e-05

Ukrayna

1.9949987500084926e-05

Almanya

1.719826508628011e-05

Türkiye

1.3758612069024087e-05

Vietnam

1.1694820258670474e-05

Fransa

4.609135043891892e-06

baba

-2.4077571120792153e-06

o

-5.933401454766638e-06

Mısır

-1.444654267247529e-05

Pakistan

-2.2472399712312217e-05

kız

-3.4052564870834615e-05

gelin

-3.439653017256022e-05

yazar

-3.590997749964032e-05

şef

-5.6410309482998755e-05

Yunanistan

-6.0767203304002135e-05

şair

-6.26016849140596e-05

İran

-7.759857206852703e-05

Kore

-7.94559846986141e-05

onun

-9.992192015128743e-05

doktor

-0.00014996887155236256

adam

-0.00016097576120758183

CEO

-0.0001726705814662523

asker

-0.00017909126709889065

Guatemala

-0.00018161367931111795

ordu

-0.00018161367931111795

diplomat

-0.00019846797909567245

işçi

-0.0002260998583338687

model

-0.0002555662191821224

sanatçı

-0.0002734524148718537

Fas

-0.0003033773961219811

abi

-0.00043477214138116116

Paraguay

-0.00043752386379496594

öğretmen

-0.0004499066146570876

Peru

-0.0004939341732779647

Singapur

-0.0005169798484935801

koruma

-0.0005242031198298177

oğlan

-0.0007691064146584464

müdür

-0.0019124470775943481


Notes

  • Higher entropy scores indicate an increase of Information Flow between the layers of the neural network. This also increases the interrelatedness between a term and its surroundings and reduces overfitting.

  • Higher is not automatically better: Depending on the base model and task specific training data, optimization at training time has equally valid reasons for reducing entropy scores.