For an atomic force microscope (AFM) system equipped with a nanosensor, an accurate varying-gain dynamic model is obtained when considering the piezoscanner bending effect, which is then utilized to design an advanced discrete-time model-predictive controller (DMPC) achieving accurate tracking performance for any given trajectory. Specifically, considering the features of the piezoscanner in the AFM system, a segmented swept signal with decreasing amplitudes is adopted as the input exerted on the piezoscanner, with the collected data utilized to setup a dynamic model based on the numerical algorithm for subspace state-space system identification (N4SID) algorithm, where the varying gain is successfully acquired by a polynomial fitting method to increase model precision. Based on the predicted dynamic behavior of the varying-gain model, an advanced DMPC algorithm is designed to fasten the system response and to enhance the robustness of the closed-loop system. The proposed modeling/control strategy is implemented and then applied to a practical AFM system, with the obtained experimental results clearly demonstrating the superior performance of the designed AFM closed-loop control system.