An a-Fair Optimization Model and Numerical Algorithms for Selecting a Common Lexical Norm of Turkic Languages

Abstract

The selection of a common lexical norm for Turkic languages is formulated as a discrete optimization problem that ensures fair intelligibility for speakers of all languages. For each concept, one form is chosen from a set of candidates; intelligibility for speakers of each language is estimated by a feature-weighted edit distance, and the choice is governed by an a-fair objective function together with phonological consistency constraints. We prove that the weighted averaging model is the special case a = 0, while the max-min criterion is the limit a → ∞. An exact mixed-integer linear programming formulation and an approximate algorithm based on relaxation, mirror ascent, rounding and local search are developed. A computational experiment on 1016 concepts of five Turkic languages from the NorthEuraLex database shows that under weighted averaging the intelligibility for the worst-off language (Kazakh) is 0.528, whereas the max-min solution raises it to 0.704 (by 33.2%) while reducing the population-weighted average by only 6.0%. The proposed algorithm deviates from the exact solution by less than 0.03% and is up to 49 times faster for a = 1.