. . . . "preprint" . . . . . " Evolver, model merging in a genetic algorithm\nImproves on current merging techniques (my beloved TIES 🫣 )\nTrain diverse models\nMerge regularly or take diff between two models\nUpdate some parameters\nKeep if good\nRepeat\n\nhttps://arxiv.org/abs/2406.12208\n@jingli9111 @banting_liu @576gsk https://twitter.com/LChoshen/status/1803410440535326786/photo/1\n\n Merging is aimed at taking many models and getting one that generalizes better, there are various methods for it, read more e.g. on TIES\nhttps://x.com/prateeky2806/status/1665759148380758022\n\n Genetic algorithms evolve models, in steps:\nCreate mutations (here new m = m_old + a(m_1-m_2)) m are models a some constant\n\nCrossover, take some of the mutation and apply it, for each parameter randomly keep m_old or update to m_new\n\nSurvive, keep only the best performing on val\n\n By sometimes merging and sometimes evolving (and dev sets) they improve over all current methods https://twitter.com/LChoshen/status/1803410445635653960/photo/1\n\n In some sense, this can be seen as a better search in the region between the merged models, which we know is not equally good but all better than the edges\nhttps://x.com/LChoshen/status/1729488495515713672 https://twitter.com/LChoshen/status/1803410447246250483/photo/1\n\n" . "TIES" . "evolver" . "genetic\\_algorithms" . "knowledge\\_fusion" . "model\\_merging" . . . "forumPost" . "forumPost" . . . . . . . . . . . . "RSA" . "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArHtI92jm8pAYVsvJabxLGfOT+7G0JyJGh2gwjB5x2pFPga6wWTd+rNBWWUZViIFnaJrBEsJpgdnoupLU9ppwn+khMiGRfxqGsDDzwHcj3Jc75CRys7d3etwXdBdoXfBgjsJiZBazwm13idr6tljRrC1TaEJBnRQAqzBw9cLDeGY77cSznzXT39feUGT168dpCSE9O6u/48DvvWVqciHGsH9cQ+LroJJVsMrorwtsdZnAK+q48wtIP6pIpw5shSJ5LnA0qeN/f4TvTFDV6ItYIXjiWWpTECc/Bxmfnyat3B5xWCu9nvz8fEs7Ns0TuzQwT3/K55iSKDEIi/E0nO97xwIDAQAB" . "DNd+uaVNO1EL2UnPQPKV4vb8L7Raa/kQy4vZM/hINGKItf80gXEU92oqDkX0iQjdWVeW1qvhQIneMk0X1opywCbxtOFfqgoWCCApVokDmjuHB9pH+iSMubN4xflRdPg2K6Vypi6WR5l4dU3VvCRRT9BICyzUXNqyca5KDAqtU2gpQOT6qMnQrKpLDev88NeCWI/F/2M9WiNeuLbZvr72bOmEmSgMANu6wZE81vyJjM45lxevTOTviZC74BZdP3RMyaBb7nRlU4Ek8Wazux8Oc4PHzMp0RV/neTirBfA+NLlx0XfWXj+q/jvepqIiSnQdm/jkf4PpZ/N3GLYp4JWyrA==" . . . . "2024-09-12T18:58:02.418Z"^^ . . . . . "CoSMO Semantic Post" . . "0xf6ECcfD463afB464dcC85b051DF2E93E2646E6D2" . . "Leshem Choshen 🤖🤗 @ICML wanna talk?" .