The former Wall Street financier had drawn criticism after calling a reporter to give a profanity-laced tirade against his colleagues.
Mr Trump’s chief of staff Reince Priebus and spokesman Sean Spicer both left their posts with his appointment.
The decision was made by Mr Trump’s new chief of staff, Gen John Kelly, who was sworn in on Monday.
The president was also unhappy with Mr Scaramucci’s performance, the White House confirmed.
Mr Trump’s spokeswoman, Sarah Sanders, said the president thought Mr Scaramucci’s comments to the reporter “were inappropriate for a person in that position”.
He has not been moved to another White House role, she added.
Mr Trump had tweeted in the morning about job and wage figures, and insisted there is “No WH chaos”, referring to reports about fighting among White House staff.
Mr Scaramucci had boasted of reporting directly to the president, rather than to his chief of staff.
Gen Kelly, who had formerly served as secretary of homeland security, was sworn in on Monday morning at the White House.