fusion.

Cancer remains one of the leading causes of mortality worldwide, and lung cancer, liver cancer, and colorectal cancer continue to contribute a substantial share of cancer deaths at both global and national scales [1], [2]. In China, national estimates for 2022 similarly indicate a heavy cancer mortality burden, with lung cancer as the leading cause of cancer death and liver and colorectal cancers also accounting for a large number of deaths [3]. Because prognosis is strongly stage-dependent, shifting detection toward earlier stages is consistently linked to improved treatment options and better outcomes across major solid tumors, including colorectal and liver cancers [4], [5], while early cancer detection is increasingly viewed as a central pillar of future cancer control strategies [6]. In routine clinical workflows, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and X-ray imaging are essential diagnostic modalities that enable screening, diagnosis, staging, and longitudinal monitoring; yet the earliest malignant lesions remain intrinsically difficult to detect even under careful expert review [7]. A core reason is that early lesions may be only a few millimeters, exhibit extremely low contrast relative to surrounding tissues, or present ambiguous boundaries, such as subsolid ground-glass opacities whose definitions and radiologic characteristics are well-recognized in thoracic imaging standards [8].

Moreover, detection difficulty increases when nodules occur adjacent to vessels or pleura, where lesion appearance can visually blend with dominant normal anatomy; this challenge has been repeatedly reflected in computer-aided detection research on pulmonary nodules, which treats variability in size and appearance as a primary driver of false negatives and false positives [9]. In volumetric CT, partial-volume averaging can mix tumor and normal tissue signals within a voxel, and the resulting loss of apparent contrast is a known image-physics limitation that becomes more pronounced under certain acquisition and reconstruction settings (e.g., thicker slices) [10]. In MRI, image interpretability can be further degraded by motion from respiration and cardiac activity, and motion artifacts are widely recognized as frequent and difficult-to-eliminate sources of error in clinical imaging [11].

Deep learning has reshaped medical image analysis by learning discriminative representations directly from pixel-level evidence and by supporting both classification and segmentation paradigms at scale [12], [13]. This progress has been reinforced by broad clinical interest in applying machine learning to medical decision-making and diagnostic workflows [14]. Architecturally, U-Net-style encoders–decoders have become a standard choice for biomedical segmentation because they can capture context while preserving localization cues [15], and 3D variants such as V-Net extend this principle to volumetric data [16]. Meanwhile, high-capacity classification backbones such as ResNet and EfficientNet provide complementary inductive biases for representation learning, with ResNet’s residual design enabling very deep feature hierarchies and EfficientNet providing a principled scaling strategy for accuracy–efficiency trade-offs [17], [18]. Despite these advances, robust early-stage detection under heterogeneous acquisition conditions remains challenging for two structural reasons that directly motivate our work.

First, early malignant findings frequently exhibit a “needle-in-a-haystack” characteristic: Without explicit guidance toward anatomically relevant regions, discriminative evidence can be diluted by dominant normal structures, causing small or low-contrast lesions to be missed. Second, relying on a single feature extractor can amplify inductive bias and encourage dataset-specific shortcuts, yielding brittle decision rules that degrade under scanner/vendor changes, protocol variations, reconstruction differences, noise conditions, and motion artifacts, an issue consistent with empirical evidence that medical imaging models can show variable generalization across institutions [19]. These observations suggest that improving early detection is not only a matter of training larger models; it requires architectural mechanisms that explicitly control where the model learns and that reduce dependence on a single representation pathway.

Motivated by these needs, the objective of this paper is to propose an architecture for robust and auditable early cancer detection across heterogeneous medical imaging modalities. We introduce a segmentation-guided dual-backbone fusion framework in which a U-Net–based module produces a probabilistic Region-of-Interest (ROI) mask to enforce ROI-focused learning, and two complementary encoders (ResNet and EfficientNet) extract deep representations in parallel that are fused through Deep Feature Concatenation to reduce single-backbone failure modes [15], [17], [18]. To support clinical usability, the framework is designed to output both calibrated malignancy probabilities and interpretable ROI evidence; calibration is emphasized because modern neural networks can be miscalibrated even when their classification accuracy is high [20]. For interpretability and auditability, the ROI/mask output provides case-level visual evidence, aligning with widely adopted gradient-based localization approaches that help characterize failure modes and dataset bias in deep models [21]. Our main work is therefore architectural: we target early-lesion evidence dilution via segmentation-guided ROI learning, and we target domain-shift brittleness via complementary feature fusion, while retaining an auditable output interface suitable for systematic miss analysis and clinical review.

In summary, the contributions of this paper are threefold: 1) an ROI-guided detection architecture that explicitly addresses early-lesion “needle-in-a-haystack” dilution by coupling segmentation-derived masks with downstream classification learning, 2) a dual-encoder fusion design that leverages complementary inductive biases (ResNet/EfficientNet) to improve robustness under heterogeneous acquisition conditions, and 3) an auditable output design that combines calibrated probabilities with interpretable spatial evidence, enabling structured error attribution and more deployment-aligned evaluation. Finally, our framing is consistent with broader evidence that deep learning can reach expert-level performance in image-based cancer classification tasks and that trustworthy deployment requires attention to generalization, calibration, and interpretability [22].

This section describes the complete architecture of the proposed segmentation-guided dual-backbone fusion framework for robust early cancer detection. The design directly addresses two core structural challenges in early-stage medical image analysis: first, the dilution of discriminative lesion evidence by dominant normal anatomical structures, and second, the instability of single-backbone models under heterogeneous acquisition conditions. To overcome these limitations, the framework integrates three tightly connected components: a segmentation module for ROI guidance, two complementary feature extraction backbones, and a deep feature fusion and prediction head. The overall workflow is illustrated in Fig. 1. An input medical image, after preprocessing, is transformed into an ROI-focused representation using a segmentation-guided mechanism. The ROI-focused image is then processed in parallel by two complementary convolutional neural network encoders: A ResNet branch and an EfficientNet branch. Each branch produces a high-level feature vector, and these feature vectors are fused through deep feature concatenation before being passed to a prediction head that outputs a calibrated malignancy probability (or a multi-class prediction, depending on the task). This parallel and fused design ensures that the final decision does not rely on a single representation pathway, thereby improving robustness under domain shift.

As shown in Fig. 1, the ROI-focused image is simultaneously fed into both encoders. The ResNet branch extracts hierarchical residual features that emphasize multi-level semantic abstraction and stable gradient propagation. The EfficientNet branch extracts compound-scaled features that balance depth, width, and resolution, providing a complementary inductive bias. The outputs of both branches are global feature vectors, which are concatenated to form a unified representation. This fused representation reduces dependence on any single backbone’s structural bias or failure mode. The ROI guidance mechanism is enabled by a U-Net–based segmentation module, illustrated in Fig. 2. This component generates a pixel-wise probability map that highlights lesion-relevant regions or anatomically meaningful structures. The U-Net architecture follows a standard encoder–decoder design with skip connections. The encoder progressively captures multi-scale contextual information by reducing spatial resolution while increasing channel depth. The decoder restores spatial resolution while integrating fine-grained details from earlier layers through skip connections. This structure is particularly effective for detecting small and low-contrast lesions, where boundary information and spatial context are both critical.

The output of the segmentation network is a probabilistic mask that identifies lesion-relevant regions. Instead of using the mask purely as a final output, the proposed architecture integrates it directly into the classification pipeline. As shown in Fig. 3, the predicted mask is used to produce an ROI-focused image through soft gating or cropping. In soft gating, the mask emphasizes lesion-relevant pixels while preserving the surrounding anatomical context. In cropping mode, the high-confidence region is extracted and resized to maintain effective resolution. In practice, soft gating is preferred because it avoids removing potentially informative contextual structures while still mitigating background dominance.

Fig. 3 illustrates how the original CT image is transformed into an ROI-focused representation using the U-Net output. The left panel shows the original CT slice, the middle panel shows the predicted segmentation mask, and the right panel shows the ROI-focused image in which lesion-relevant regions are emphasized. This mechanism directly addresses the early-lesion “needle-in-a-haystack” problem: Without spatial guidance, small malignant regions may be overwhelmed by dominant normal tissue patterns. By enforcing ROI-focused learning, the model is explicitly guided toward clinically meaningful regions. After ROI focusing, the image is passed to the dual-backbone feature extraction module. The ResNet branch emphasizes deep hierarchical representation learning with residual connections, which stabilize optimization and enable effective gradient propagation. The EfficientNet branch emphasizes balanced scaling of network depth, width, and input resolution, which improves efficiency and generalization. Because these architectures rely on different structural priors, their learned feature representations are complementary. The concatenation of their feature vectors forms a unified embedding that integrates structural boundary-sensitive information and high-level semantic information.

The fused representation is then forwarded to a lightweight fully connected prediction head. This head produces a malignancy probability for binary detection or a probability distribution for multi-class tasks. To support deployment in clinical decision support systems, the architecture is designed to allow probability calibration, ensuring that predicted confidence scores align with empirical risk levels. Importantly, the architecture produces two interpretable outputs: the ROI mask and the classification probability. This dual-output design supports structured error analysis. If a false negative occurs and the ROI mask fails to localize the lesion, the failure can be attributed to segmentation uncertainty. If the ROI mask is accurate but classification is incorrect, the error can be attributed to representation or decision-level limitations. This separation enhances interpretability and supports systematic model refinement.

Therefore, the proposed architecture integrates segmentation-guided spatial focusing with dual-backbone feature fusion into a unified and deployable pipeline. The segmentation module controls where the model learns, reducing background-driven evidence dilution. The dual-backbone design controls how the model represents features, reducing sensitivity to single-architecture bias and domain shift. The final fusion and calibrated prediction head provide stable and interpretable outputs suitable for early-stage cancer detection in heterogeneous real-world imaging environments.

Let the imaging modality be m ∈ {CT, MRI, Xray}. For each modality, we define a dataset.

(m) (m) (m) Nm

= {(X , yi, s )} . (1)

i i i=1

(m) H×W×C

Here, x ∈ ℝ denotes an image (or slice/patch) acquired under modality m, where H and W represent spatial resolution and c denotes the number of channels (e.g., single-channel grayscale for many radiological inputs, or multi-channel representations produced by windowing or multi-sequence MRI concatenation). The label yi is the clinical target for classification, which can be formulated as binary malignancy detection (yi ∈ {0,1}) or multi-class lesion categorization (yi ∈ {1, … , K}) depending on the dataset definition and clinical use

(m) H×W

case. The variable s ∈ {0,1} is a segmentation ground-truth mask when available, typically indicating lesion pixels (and, in some datasets, organ regions that provide an anatomical context for lesion interpretation). This formulation accommodates both 2D settings (X-ray images, CT/MRI slices, histopathology patches) and 3D settings (CT/MRI volumes) by representing a volume either as a stack of slices or by extracting clinically meaningful 2D/2.5D views, while keeping the notation consistent for the architecture description.

A central practical constraint in medical imaging is that segmentation labels are not uniformly available across datasets or institutions. Pixel-level annotation is expensive and requires expert labor, and in early-stage cases it can be especially ambiguous due to low contrast, small lesion size, and boundary uncertainty. Therefore, the learning setting is inherently “partially supervised” with respect to segmentation: for some datasets, s exists and can be used for supervised segmentation training, while for other datasets only the classification label yi is available. The proposed framework is designed for this mixed-label regime by separating the roles of segmentation supervision and ROI guidance. Specifically, the framework 1) trains the segmentation module using datasets that provide reliable masks, and 2) applies the predicted masks as ROI guidance even when training the classifier on label-only datasets. When the segmentation module is not directly trainable on a given dataset (because masks are absent), weak localization strategies (e.g., CAM-style heatmaps or attention pooling) can be incorporated as an auxiliary mechanism to approximate lesion-relevant regions and reduce background dominance; this ensures that ROI-focused learning remains feasible without requiring universal pixel-level labeling.

Given an input image x, the system produces two outputs that serve complementary roles. The first output is a malignancy probability p̂ (or, for multi-class settings, a probability vector p̂ ), which is used for triage, screening, or downstream decision support. The second output is an interpretable ROI representation, either as a predicted binary mask ŝ or as a probabilistic mask p prior to thresholding. This dual-output structure is not only an explainability feature; it is also a functional component of the learning objective because ROI guidance directly shapes which pixels contribute most to representation learning. In early cancer detection, where discriminative patterns are sparse, such explicit spatial guidance is intended to mitigate the “evidence dilution” phenomenon in which dominant normal structures overwhelm lesion cues in the learned embedding space.

The primary objective of this work is robust early-stage detection under heterogeneous acquisition conditions. Robustness here refers to maintaining reliable discrimination when acquisition factors change, including scanner vendor differences, protocol variations, reconstruction settings (e.g., slice thickness in CT), noise and contrast variability, and motion artifacts. Early-stage detection emphasizes reducing false negatives for small or low-contrast lesions, which implies that model selection and evaluation must be aligned with sensitivity-critical metrics rather than solely overall accuracy.

Consequently, the evaluation focus is on recall and F1-score, supported by AUC to capture threshold-independent discrimination, while explicitly quantifying degradation under domain shift by separating in-domain and out-of-domain performance. From a design perspective, the goal is to achieve a detection pipeline that reduces missed early lesions while preserving stable performance across domains, and that provides auditable intermediate evidence (ROI/mask) to support systematic error attribution. Concretely, when a false negative occurs, the presence of the ROI output enables distinguishing whether the failure is driven by ROI/localization error (e.g., the lesion region was not highlighted under low contrast or motion artifacts) or by classification/representation error (e.g., the ROI is plausible but the classifier remains uncertain), which is essential for iterative refinement and safety-oriented deployment.

Finally, the formulation in (1) naturally supports two deployment-relevant learning modes. In the first mode, training and inference are performed per modality, yielding modality-specific models that share the same architectural template and evaluation protocol. In the second mode, when paired multi-modal data exist for the same patient (e.g., CT and MRI acquired for complementary diagnostic purposes), the framework can be extended to multi-modal fusion by learning modality-specific encoders and combining their embeddings at a fusion stage; importantly, the base formulation still holds because each modality contributes an instance

(m) (m)

(X , yi, s ), and missing modalities can be handled by falling back to the available modality pathways. This design keeps the problem statement consistent while allowing practical flexibility across different clinical data availability scenarios.

The proposed framework is a segmentation-guided, dual-backbone fusion pipeline composed of preprocessing, ROI generation, ROI focusing, parallel feature extraction, feature fusion, and prediction.

When segmentation labels are available, the segmentation branch is trained under an objective that explicitly addresses the extreme class imbalance typical of early lesions, where lesion pixels may constitute only a very small fraction of the field of view. In this setting, overlap-based losses are preferred because they measure region agreement rather than pixel-wise accuracy dominated by background. We therefore adopt a composite segmentation loss that balances a Dice-type term, which promotes spatial overlap between the predicted mask and the ground truth, with a pixel-wise term such as Binary Cross-Entropy (BCE) or Cross-Entropy (CE), which penalizes local misclassification and stabilizes optimization near boundaries.

Concretely, Dice loss discourages trivial all-background solutions and is robust to lesion sparsity, while BCE/CE improves local calibration of the probability map and helps refine fine-grained edges in ambiguous, low-contrast regions. In practice, this combination is particularly important for early-stage lesions, where boundaries may be partially missing due to partial-volume effects, motion blur, or protocol-dependent contrast, and where relying on a single loss often yields either overly smooth masks (overemphasis on overlap) or noisy masks (overemphasis on pixel-wise terms). For the classification branch, the model is trained to output either a malignancy probability for binary detection or a probability vector for multi-class lesion categorization. BCE is used for detection tasks because it directly optimizes probabilistic separation between malignant and non-malignant cases. For multi-class settings, categorical CE is used to optimize the likelihood of the correct class under a softmax output. Because early detection datasets are frequently imbalanced, often with far fewer positive or early-stage samples than negative or benign samples, class weighting (or equivalent imbalance-aware sampling) is incorporated to prevent the classifier from converging to a majority-class-biased decision rule.

In addition, the classification objective can be supplemented with label-smoothing or focal-style reweighting when hard negatives dominate, although the core framework remains compatible with standard CE/BCE to keep the training pipeline reproducible and easily deployable. The overall training is formulated as a multi-task objective that couples classification learning with segmentation learning whenever masks exist. The joint objective is written as

ℒ = ℒcls + βℒseg, (1)

where ℒcls denotes the classification loss (BCE for binary detection or CE for multi-class tasks), ℒseg denotes the segmentation loss (e.g., Dice+BCE/CE), and β controls the relative contribution of segmentation supervision. The role of β is not merely numerical balancing; it acts as a structural coupling coefficient that determines how strongly ROI learning is “anchored” by pixel-level supervision. When β is too small, ROI guidance may become weak, and the classifier may revert toward global-context shortcuts; when β is too large, training may over-prioritize mask fidelity at the expense of classification discrimination, particularly if segmentation labels are noisy or inconsistent across datasets. Accordingly, β is selected on the validation set using sensitivity-oriented criteria (e.g., recall/F1 and AUC), ensuring that ROI guidance improves early-lesion detection rather than only producing visually plausible masks.

In mixed-label settings, where some datasets provide (x, y, s)and others provide only (x, y), training proceeds in an alternating or hybrid-batch manner to avoid discarding label-only data while still benefiting from segmentation supervision where available. Specifically, segmentation-labeled batches are used to update the segmentation parameters θs (and, if the implementation shares early features, any shared layers), while classification-labeled batches are used to update the classifier parameters θR, θE,θC. Importantly, ROI guidance is still applied in classification-only batches by feeding the input through the current segmentation model to produce a predicted probabilistic mask and using that mask for ROI focusing. This design ensures that ROI-focused learning remains active across the full training distribution, even when explicit masks are absent, and it reflects the practical annotation landscape in medical imaging, where segmentation is often sparse but case-level labels are more widely available. If the predicted ROI is uncertain for some samples (for example, due to domain shift), the framework can use soft gating rather than hard thresholding to preserve contextual cues and reduce catastrophic ROI failures, while still encouraging lesion-centric representation learning.

Optimization is performed using Adam or AdamW, which are well-suited to multi-component deep networks and typically yield stable convergence under heterogeneous datasets. Early stopping and model selection are performed using a validation protocol that prioritizes early-detection objectives, emphasizing recall, F1, and AUC rather than accuracy alone. This choice aligns the training dynamics with the clinical risk profile: in early-stage cancer detection, the cost of false negatives is usually higher than the cost of additional follow-up review triggered by false positives. In addition, because the system is designed for decision support, calibration can be applied as a post-training step on the validation set (for example, temperature scaling) so that the predicted probabilities better reflect empirical risk and can be thresholded in a clinically meaningful manner.

This section reports a fully reproducible synthetic evaluation to demonstrate the calculations, reporting structure, and robustness analyses that follow the proposed framework. The synthetic results are presented as an implementation-level “dry run” for the architecture and evaluation protocol; they are not clinical evidence and should not be interpreted as medical performance. All models were evaluated under identical data splits and decision thresholds, and the reporting emphasizes early-stage sensitivity and robustness under acquisition heterogeneity.

This paper presented a complete segmentation-guided dual-backbone fusion framework for robust early cancer detection across heterogeneous medical imaging modalities. The proposed pipeline addresses the dominant early-stage failure modes by explicitly enforcing ROI-focused learning through a U-Net–derived mask and by reducing single-backbone brittleness via complementary ResNet/EfficientNet feature extraction and deep feature concatenation fusion. The evaluation section illustrated a clear, reproducible reporting protocol with full metric calculations, emphasizing early-lesion sensitivity (recall/F1 and size-stratified analysis), robustness under acquisition domain shift, calibration for decision-support readiness, and auditability through ROI-based miss attribution.

Despite these strengths, the current study’s quantitative demonstration is based on synthetic data and therefore does not constitute clinical evidence; additionally, the framework’s performance may be bounded by segmentation reliability when pixel-level masks are scarce or when domain shift degrades ROI quality. Future work will validate the system on multi-institutional clinical datasets with patient-level leakage control, extend robustness via domain adaptation and uncertainty-aware ROI gating (especially under motion artifacts and protocol shifts), and enhance clinical usability by integrating radiologist-in-the-loop error review, operating-point optimization for screening workloads, and comprehensive calibration reporting (e.g., ECE/Brier with post-hoc scaling) to support safe deployment in real-world decision-support workflows.

Nafei: conceptualization, methodology, software, formal analysis, investigation, resources, data maintenance, writing-creating the initial design, writing-reviewing and editing. Chen: validation, writing, reviewing, and editing, visualization, funding procurement, and methodology. All authors have read and agreed to the published version of the manuscript.

This research received no external funding.

The data used in this study are available from the corresponding author upon reasonable request.

The authors declare no conflict of interest.