Document Type
Article
Publication Date
1-1-2025
Original Citation
Niyonkuru E,
Gomez M,
Casarighi E,
Antogiovanni S,
Blau H,
Reese J,
Valentini G,
Robinson P.
Replacing non-biomedical concepts improves embedding of biomedical concepts. PLoS One. 2025;20(5):e0322498.
Keywords
JGM, Algorithms, Machine Learning, Semantics, Medical Informatics Computing
JAX Source
PLoS One. 2025;20(5):e0322498.
ISSN
1932-6203
PMID
40324016
DOI
https://doi.org/10.1371/journal.pone.0322498
Grant
This work was supported by the National Institutes of Health (NIH) Office of the Director 5R24OD011883. PNR received additional support from the Alexander von Humboldt foundation.
Abstract
Embeddings are semantically meaningful representations of words in a vector space, commonly used to enhance downstream machine learning applications. Traditional biomedical embedding techniques often replace all synonymous words representing biological or medical concepts with a unique token, ensuring consistent representation and improving embedding quality. However, the potential impact of replacing non-biomedical concept synonyms has received less attention. Embedding approaches often employ concept replacement to replace concepts that span multiple words, such as non-small-cell lung carcinoma, with a single concept identifier (e.g., D002289). Also, all synonyms of each concept are merged into the same identifier. Here, we additionally leveraged WordNet to identify and replace sets of non-biomedical synonyms with their most common representatives. This combined approach aimed to reduce embedding noise from non-biomedical terms while preserving the integrity of biomedical concept representations. We applied this method to 1,055 biomedical concept sets representing molecular signatures or medical categories and assessed the mean pairwise distance of embeddings with and without non-biomedical synonym replacement. A smaller mean pairwise distance was interpreted as greater intra-cluster coherence and higher embedding quality. Embeddings were generated using the Word2Vec algorithm applied to a corpus of 10 million PubMed abstracts. Our results demonstrate that the addition of non-biomedical synonym replacement reduced the mean intra-cluster distance by an average of 8%, suggesting that this complementary approach enhances embedding quality. Future work will assess its applicability to other embedding techniques and downstream tasks. Python code implementing this method is provided under an open-source license.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.