In this paper, we propose an adaptive algorithm for L1-fidelity color image restoration by using saturation-value total variation. The main contribution of this paper is to employ the generalized cross validation method efficiently and automatically to estimate the regularization parameter in a saturation-value total variation plus L1-fidelity color image restoration model. We consider Poisson noise and mixed noise in this paper, and the experimental results show that the visual quality and the SSIM/PSNR/SAM values of the restored images by using the proposed algorithm are competitive with other tested existing methods, which makes the proposed algorithm to be comparable both quantitatively and qualitatively.