research-article
Authors: Mengqiu Shao, Peng Xue, Xi Zhou, and Xiao Shen
Volume 152, Issue C
Published: 17 July 2024 Publication History
- 0citation
- 0
- Downloads
Metrics
Total Citations0Total Downloads0Last 12 Months0
Last 6 weeks0
New Citation Alert added!
This alert has been successfully added and will be sent to:
You will be notified whenever a record that you have chosen has been cited.
To manage your alert preferences, click on the button below.
Manage my Alerts
New Citation Alert!
Please log in to your account
- View Options
- References
- Media
- Tables
- Share
Highlights
•
Class-aware contrastive domain adaptation to mitigate intra-class domain discrepancy and enlarge inter-class domain discrepancy.
•
Positive and negative pseudo-labeling to specify the presence and absence of a specific class in a target node.
•
Potential supervised information from more target nodes can be exploited to facilitate class-aware domain alignment.
Abstract
The performance of graph Neural Network (GNNs) can degrade significantly when trained on the graphs with noisy edges connecting nodes from different classes. To mitigate the negative effect of noisy edges, previous studies have largely focused on predicting the label agreement between node pairs within a single network. So far, predicting noisy edges across different networks remains largely underexplored. To bridge this gap, our work studies a novel problem of cross-network homophilous and heterophilous edge classification (CNHHEC), aiming to predict the label agreement of edges in an unlabeled target network by transferring the knowledge from a labeled source network. A novel Contrastive Domain-adaptive Graph Self-training Network (CDGSN) is proposed. Firstly, CDGSN learns node and edge embeddings end-to-end by a GNN encoder with adaptive edge weights during neighborhood aggregation. Secondly, to facilitate knowledge transfer across networks, CDGSN employs an adversarial domain adaptation module to align edge embeddings across networks, and also designs a novel contrastive domain adaptation module to conduct class-aware cross-network alignment of node embeddings. As a result, the intra-class domain divergence can be mitigated while the inter-class domain discrepancy can be enlarged to yield network-invariant and label-discriminative node and edge embeddings. Moreover, CDGSN designs a selective positive and negative pseudo-labeling strategy to assign positive (negative) pseudo-labels to the target nodes with extremely high (low) prediction confidence of belonging to each specific class. Such pseudo-labeled target nodes would be employed to iteratively re-train the model in a self-training manner, so as to obtain more reliable target pseudo-labels progressively to guide class-aware domain alignment. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed CDGSN on the CNHHEC problem. The performance of CDGSN is robust against various types of edge embeddings. When adopting three operators to construct edge embeddings, the proposed CDGSN can improve the state-of-the-art method for CNHHEC by an average of 0.5 %, 2.8 %, and 10.8 % in terms of AUC, and 1.3 %, 3.0 %, and 6.3 % in terms of AP, respectively.
References
[1]
Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, S.Y. Philip, A comprehensive survey on graph Neural Network, IEEE Trans Neur. Net. Learn. Syst. 32 (1) (2020) 4–24.
[2]
B. Li, Y. Zhang, C. Zhang, X. Piao, Y. Hu, B. Yin, Multi-scale hypergraph-based feature alignment network for cell localization, Pattern Recognit. 149 (2024).
[3]
B. Lei, Y. Zhu, S. Yu, H. Hu, Y. Xu, G. Yue, T. Wang, C. Zhao, S. Chen, P. Yang, Multi-scale enhanced graph convolutional network for mild cognitive impairment detection, Pattern Recognit. 134 (2023).
[4]
T.N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, in: International Conference on Learning Representations, 2016.
[5]
P. Zheng, X. Guo, E. Chen, L. Qi, L. Guan, Edge-labeling based modified gated graph network for few-shot learning, Pattern Recognit. 150 (2024).
[6]
H. Yang, X. Yan, X. Dai, Y. Chen, J. Cheng, Self-enhanced gnn: improving graph Neural Network using model outputs, in: 2021 International Joint Conference on Neural Network (IJCNN), 2021, pp. 1–8.
[7]
E. Dai, W. Jin, H. Liu, S. Wang, Towards robust graph Neural Network for noisy graphs with sparse labels, in: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 2022, pp. 181–191.
[8]
H. Chen, Y. Xu, F. Huang, Z. Deng, W. Huang, S. Wang, P. He, Z. Li, Label-aware graph convolutional networks, in: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020, pp. 1977–1980.
[9]
E. Dai, C. Aggarwal, S. Wang, Nrgnn: learning a label noise resistant graph neural network on sparsely and noisily labeled graphs, in: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 227–236.
[10]
X. Shen, M. Shao, S. Pan, L.T. Yang, X. Zhou, Domain-adaptive graph attention-supervised network for cross-network edge classification, IEEE Trans. Neur. Net. Learn. Syst. (2023).
[11]
Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, V. Lempitsky, Domain-adversarial training of neural network, J. Mach. Learn. Res. 17 (1) (2016) 2096-2030.
[12]
G. Kang, L. Jiang, Y. Yang, A.G. Hauptmann, Contrastive adaptation network for unsupervised domain adaptation, in: Proceedings of the Conference on Computer Vision and Pattern Recognition, 2019, pp. 4893–4902.
[13]
P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio, Graph attention networks, in: International Conference on Learning Representations, 2018.
[14]
Y. Kim, J. Yim, J. Yun, J. Kim, Nlnl: negative learning for noisy labels, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 101–110.
[15]
C. Liu, J. Wu, W. Liu, W. Hu, Enhancing graph Neural Network by a high-quality aggregation of beneficial information, Neur. Net. 142 (2021) 20–33.
[16]
O. Stretcu, K. Viswanathan, D. Movshovitz-Attias, E. Platanios, S. Ravi, A. Tomkins, Graph agreement models for semi-supervised learning, Adv. Neur. Inf. Proc. Syst. (2019) 8713–8723.
[17]
Y. Yang, Y. Sun, F. Ju, S. Wang, J. Gao, B. Yin, Multi-graph fusion graph convolutional networks with pseudo-label supervision, Neur. Net. 158 (2023) 305–317.
[18]
Z. Lan, Y. Ma, L. Yu, L. Yuan, F. Ma, AEDNet: adaptive edge-deleting network for subgraph matching, Pattern Recognit. 133 (2023).
[19]
K. Zhao, Z. Zhang, B. Jiang, J. Tang, LGLNN: label guided graph learning-neural network for few-shot learning, Neur. Net. 155 (2022) 50–57.
[20]
Z. Lu, Y. Peng, Exhaustive and efficient constraint propagation: a graph-based learning approach and its applications, Int. J. Comput. Vis. 103 (3) (2013) 306–325.
[21]
A. Gretton, K. Borgwardt, M. Rasch, B. Schölkopf, A. Smola, A kernel method for the two-sample-problem, in: Advances in Neural Information Processing Systems, 2007, pp. 513–520.
[22]
G. Kang, L. Jiang, Y. Wei, Y. Yang, A. Hauptmann, Contrastive adaptation network for single-and multi-source domain adaptation, IEEE Trans. Pattern Anal. Mach. Intell. 44 (4) (2020) 1793–1804.
[23]
M. Long, J. Wang, G. Ding, J. Sun, P.S. Yu, Transfer feature learning with joint distribution adaptation, in: Proceedings of the IEEE International Conference on Computer Vision, 2013, pp. 2200–2207.
[24]
M. Long, Z. Cao, J. Wang, M.I. Jordan, Conditional adversarial domain adaptation, Adv. Neur. Inf. Proc. Syst. 31 (2018) 1640–1650.
[25]
A. Singh, Clda: contrastive learning for semi-supervised domain adaptation, Adv. Neur. Inf. Proc. Syst. (2021) 5089–5101.
[26]
R. Wang, Z. Wu, Z. Weng, J. Chen, G.J. Qi, Y.G. Jiang, Cross-domain contrastive learning for unsupervised domain adaptation, IEEE Trans. Multimed. (2022).
[27]
M.N. Rizve, K. Duarte, Y.S. Rawat, M. Shah, In defense of pseudo-labeling: an uncertainty-aware pseudo-label selection framework for semi-supervised learning, in: International Conference on Learning Representations, 2021.
[28]
X. Shen, Q. Dai, S. Mao, F.-l. Chung, K.S. Choi, Network together: node classification via cross-network deep network embedding, IEEE Trans. Neur. Netw. Learn. Syst. 32 (5) (2021) 1935–1948.
[29]
H. Wu, L. Tian, Y. Wu, J. Zhang, M.K. Ng, J. Long, Transferable graph auto-encoders for cross-network node classification, Pattern Recognit. (2024).
[30]
X. Shen, Q. Dai, F.-l. Chung, W. Lu, K.S. Choi, Adversarial deep network embedding for cross-network node classification, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2020, pp. 2991–2999.
[31]
X. Shen, S. Pan, K.S. Choi, X. Zhou, Domain-adaptive message passing graph neural network, Neur. Net. 164 (2023) 439–454.
[32]
Y. Zhang, C. Shi, X. Li, Z. Zhang, X. Hu, Multi-component similarity graphs for cross-network node classification, IEEE Transac. Artif. Intellig. (2023).
[33]
Y. Zhang, G. Song, L. Du, S. Yang, Y. Jin, Dane: domain adaptive network embedding, in: International Joint Conference on Artificial Intelligence, 2019, pp. 4362–4368.
[34]
Q. Dai, X. Shen, X.M. Wu, D. Wang, Graph transfer learning via adversarial domain adaptation with graph convolution, IEEE Trans. Knowl. Data Eng. (2022).
[35]
M. Wu, S. Pan, C. Zhou, X. Chang, X. Zhu, Unsupervised domain adaptive graph convolutional networks, in: Proceedings of The Web Conference 2020, 2020, pp. 1457–1467.
[36]
X. Zhang, Y. Du, R. Xie, C. Wang, Adversarial separation network for cross-network node classification, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 2618–2626.
[37]
J. Xiao, Q. Dai, X. Xie, Q. Dou, K.W. Kwok, J. Lam, Domain adaptive graph infomax via conditional adversarial networks, IEEE Transac. Net. Sci. Engineer. 10 (1) (2022) 35–52.
[38]
J. Shen, Y. Qu, W. Zhang, Y. Yu, Wasserstein distance guided representation learning for domain adaptation, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2018, pp. 4058–4065.
[39]
X. Shen, F.L. Chung, Deep network embedding for graph representation learning in signed networks, IEEE Trans. Cybern. 50 (4) (2018) 1556–1568.
[40]
T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, in: International Conference on Machine Learning, 2020, pp. 1597–1607.
[41]
T.N. Kipf and M. Welling, "Variational graph auto-encoders," arXiv preprint arXiv:1611.07308 , 2016.
[42]
G. Cui, J. Zhou, C. Yang, Z. Liu, Adaptive graph encoder for attributed graph embedding, in: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 976–985.
[43]
D. Kim, A. Oh, How to find your friendly neighborhood: graph attention design with self-supervision, in: International Conference on Learning Representations, 2021.
Recommendations
- Supervised contrastive learning for graph representation enhancement
Abstract
Graph Neural Networks (GNNs) have exhibited significant success in various applications, but they face challenges when labeled nodes are limited. A novel self-supervised learning paradigm has emerged, enabling GNN training without labeled nodes ...
Read More
- Domain-adaptive message passing graph neural network
Abstract
Cross-network node classification (CNNC), which aims to classify nodes in a label-deficient target network by transferring the knowledge from a source network with abundant labels, draws increasing attention recently. To address CNNC, we propose ...
Read More
- Informative pseudo-labeling for graph neural networks with few labels
Abstract
Graph neural networks (GNNs) have achieved state-of-the-art results for semi-supervised node classification on graphs. Nevertheless, the challenge of how to effectively learn GNNs with very few labels is still under-explored. As one of the ...
Read More
Comments
Information & Contributors
Information
Published In
Pattern Recognition Volume 152, Issue C
Aug 2024
527 pages
ISSN:0031-3203
Issue’s Table of Contents
Elsevier Ltd.
Publisher
Elsevier Science Inc.
United States
Publication History
Published: 17 July 2024
Author Tags
- Graph neural network
- Cross-network edge classification
- Graph Domain Adaptation
- Pseudo-labeling
Qualifiers
- Research-article
Contributors
Other Metrics
View Article Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
Total Citations
Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Other Metrics
View Author Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in
Full Access
Get this Publication
Media
Figures
Other
Tables
