Abstract: A lightweight network named MSAL−Unet, based on an improved U−Net, was proposed to address the challenge of balancing accuracy and computational efficiency in pavement crack detection. First, a lightweight encoder was constructed and integrated with the convolutional block attention module (CBAM) to dynamically enhance feature responses in key crack regions, thereby improving the model’s perception capability for low-contrast targets. Second, a Multi-Scale Dilated Convolution (MSDC) module was designed to expand the receptive field without reducing spatial resolution, effectively capturing multi-scale information from fine cracks to broad contextual features and significantly suppressing false positives. Furthermore, a feature pyramid network (FPN) was introduced to achieve cross-level semantic alignment and multi-scale feature fusion, enhancing the coherence of crack localization and boundary smoothness. Comparative and ablation experiments were conducted on the public dataset CRACK500 to validate the performance of the proposed method. The results demonstrate that the proposed method achieves a mean intersection over union (mIoU) of 0.8320 and a precision of 0.9180 (the highest among all compared models), with an F1−score of 0.9063, while maintaining a real-time inference speed of 34.45 frames per second. The comprehensive performance is significantly superior to mainstream lightweight networks such as BiSeNet, Fast−SCNN, SegFormer, and TransUNet. Particularly in scenarios involving faint and complex cracks, the segmentation results of MSAL−Unet exhibit optimal performance in both completeness and boundary accuracy. By synergistically optimizing multi-scale feature fusion, attention mechanisms, and lightweight structural design, this study effectively addresses the challenging balance between accuracy and efficiency in pavement crack detection, thereby providing highly reliable technical support with low computational overhead for practical road intelligent inspection systems.