Nanopublications

@prefix this: <https://w3id.org/np/RAKmi8CK-UdWMP9L6Q9YymxVSFsIlM1cP-6WDc4T4ocN0> .
@prefix sub: <https://w3id.org/np/RAKmi8CK-UdWMP9L6Q9YymxVSFsIlM1cP-6WDc4T4ocN0#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix np: <http://www.nanopub.org/nschema#> .
@prefix npx: <http://purl.org/nanopub/x/> .
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix prov: <http://www.w3.org/ns/prov#> .
@prefix schema: <https://schema.org/> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix orcid: <https://orcid.org/> .

sub:head {
  this: np:hasAssertion sub:assertion ;
    np:hasProvenance sub:provenance ;
    np:hasPublicationInfo sub:pubinfo ;
    a np:Nanopublication .
}

sub:assertion {
  <https://arxiv.org/abs/2406.12208> <https://sense-nets.xyz/hasZoteroItemType> "preprint" .
  sub:assertion dcterms:creator <https://w3id.org/np/RAoSadUw99CeqDlR2400018nqTzR_38fT86OrTzk16Vts> ;
    <http://purl.org/spar/cito/discusses> <https://x.com/LChoshen/status/1729488495515713672> , <https://x.com/prateeky2806/status/1665759148380758022> ;
    <http://purl.org/spar/cito/reviews> <https://arxiv.org/abs/2406.12208> ;
    rdfs:comment """ Evolver, model merging in a genetic algorithm
Improves on current merging techniques (my beloved TIES 🫣 )
Train diverse models
Merge regularly or take diff between two models
Update some parameters
Keep if good
Repeat

https://arxiv.org/abs/2406.12208
@jingli9111 @banting_liu @576gsk https://twitter.com/LChoshen/status/1803410440535326786/photo/1

Merging is aimed at taking many models and getting one that generalizes better, there are various methods for it, read more e.g. on TIES
https://x.com/prateeky2806/status/1665759148380758022

Genetic algorithms evolve models, in steps:
Create mutations (here new m = m_old + a(m_1-m_2)) m are models a some constant

Crossover, take some of the mutation and apply it, for each parameter randomly keep m_old or update to m_new

Survive, keep only the best performing on val

By sometimes merging and sometimes evolving (and dev sets) they improve over all current methods https://twitter.com/LChoshen/status/1803410445635653960/photo/1

In some sense, this can be seen as a better search in the region between the merged models, which we know is not equally good but all better than the edges
https://x.com/LChoshen/status/1729488495515713672 https://twitter.com/LChoshen/status/1803410447246250483/photo/1

""" ;
    schema:keywords "TIES" , "evolver" , "genetic\\_algorithms" , "knowledge\\_fusion" , "model\\_merging" ;
    <https://sense-nets.xyz/endorses> <https://x.com/LChoshen/status/1729488495515713672> ;
    <https://sense-nets.xyz/summarizes> <https://arxiv.org/abs/2406.12208> .
  <https://x.com/LChoshen/status/1729488495515713672> <https://sense-nets.xyz/hasZoteroItemType> "forumPost" .
  <https://x.com/prateeky2806/status/1665759148380758022> <https://sense-nets.xyz/hasZoteroItemType> "forumPost" .
}

sub:pubinfo {
  sub:sig npx:hasAlgorithm "RSA" ;
    npx:hasPublicKey "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArHtI92jm8pAYVsvJabxLGfOT+7G0JyJGh2gwjB5x2pFPga6wWTd+rNBWWUZViIFnaJrBEsJpgdnoupLU9ppwn+khMiGRfxqGsDDzwHcj3Jc75CRys7d3etwXdBdoXfBgjsJiZBazwm13idr6tljRrC1TaEJBnRQAqzBw9cLDeGY77cSznzXT39feUGT168dpCSE9O6u/48DvvWVqciHGsH9cQ+LroJJVsMrorwtsdZnAK+q48wtIP6pIpw5shSJ5LnA0qeN/f4TvTFDV6ItYIXjiWWpTECc/Bxmfnyat3B5xWCu9nvz8fEs7Ns0TuzQwT3/K55iSKDEIi/E0nO97xwIDAQAB" ;
    npx:hasSignature "DNd+uaVNO1EL2UnPQPKV4vb8L7Raa/kQy4vZM/hINGKItf80gXEU92oqDkX0iQjdWVeW1qvhQIneMk0X1opywCbxtOFfqgoWCCApVokDmjuHB9pH+iSMubN4xflRdPg2K6Vypi6WR5l4dU3VvCRRT9BICyzUXNqyca5KDAqtU2gpQOT6qMnQrKpLDev88NeCWI/F/2M9WiNeuLbZvr72bOmEmSgMANu6wZE81vyJjM45lxevTOTviZC74BZdP3RMyaBb7nRlU4Ek8Wazux8Oc4PHzMp0RV/neTirBfA+NLlx0XfWXj+q/jvepqIiSnQdm/jkf4PpZ/N3GLYp4JWyrA==" ;
    npx:hasSignatureTarget this: ;
    npx:singedBy <https://sense-nets.xyz/> ;
    prov:wasAssociatedWith <https://w3id.org/np/RAoSadUw99CeqDlR2400018nqTzR_38fT86OrTzk16VtssigningDelegation> .
  this: dcterms:created "2024-09-12T18:58:02.418Z"^^xsd:dateTime ;
    dcterms:creator <https://w3id.org/np/RAoSadUw99CeqDlR2400018nqTzR_38fT86OrTzk16Vts> ;
    dcterms:license <https://creativecommons.org/licenses/by/4.0/> ;
    npx:hasNanopubType <https://sense-nets.xyz/SemanticPost> ;
    npx:wasCreatedAt <https://sense-nets.xyz/> ;
    rdfs:label "CoSMO Semantic Post" ;
    prov:wasAttributedTo orcid:0000-0002-0085-6496 ;
    <https://sense-nets.xyz/hasRootSigner> "0xf6ECcfD463afB464dcC85b051DF2E93E2646E6D2" .
  <https://w3id.org/np/RAoSadUw99CeqDlR2400018nqTzR_38fT86OrTzk16Vts> foaf:account orcid:0000-0002-0085-6496 ;
    foaf:name "Leshem Choshen 🤖🤗 @ICML wanna talk?" .
}