The impurity removal process of the rare earth (RE) leachate is usually accompanied by the formation of flocs, and a certain amount of polyacrylamide flocculant needs to be added in this process. However, few studies have investigated the effect of the flocculant on the impurity removal process. Therefore, this paper compares the influence on the process of removing impurities with or without the adding of polyacrylamide (PAM). The results showed that the addition of PAM had little effect on the removal rate of impurities, but it was conducive to the recovery of RE. When ammonium bicarbonate was firstly adopted to regulate the solution pH to 5.0, and then sodium sulfide was used to adjust the pH to 5.2, and 0.5 mL/100 mL of PAM was added to the leachate, 96% Al3+ and 98% Zn2+ were removed from the leachate, while 95% RE was maintained. In addition, PAM could accelerate the settlement of flocs and then shorten the production cycle. The increase in RE recovery rate may be due to the addition of PAM to shorten the settling time of the flocs, thereby reducing the RE ions adsorbed during the flocculation process and increasing the recovery rate of the RE.