If you’ve ran the command with sudo it might have changed permissions on your cache directory.
sudo chown -R pi /home/pi/.cache
and then try the build again. Just make sure all of the directories are owned by the pi user that you are running as.