The 1911 painting, Othello and Desdemona, was taken from Ernest and Rose Heller's flat with more than a dozen other works of art and jewellery.
The Chagall was recovered last year after a Washington, DC gallery owner twice refused to purchase the painting.
He told the seller to contact the authorities for proof of ownership.
"We took the case from there," said Special Agent Marc Hess, a member of the FBI Art Crime Team.
The Maryland man who tried to sell the work had stored it in his attic for years in a customised box he fashioned out of a door jamb and plywood, according to Mr Hess.
He was given the painting in the late 1980s or early 1990s from another man who is suspected of taking it from the Upper East Side building where the Hellers lived, prosecutors said in a court filing on Thursday.
The couple owned works of art by Renoir, Picasso, Hopper and Chagall, the bureau said.
Mr Heller was a retired jewel importer who died in 1998 and his wife passed away in 2003.
The Maryland man, who has not been named, said he planned to sell the stolen Chagall to a potential buyer, but the deal fell through after a dispute over his cut of the earnings, the FBI said. Instead, he kept the painting in his attic and attempted to sell it in 2011 and again in 2017.
The painting, which depicts Shakespeare's Othello holding a sword as he gazes down at his bride, Desdemona, was still marked with the names of its owners, "Mr + Mrs ES Heller, New York", when the Maryland man tried to sell it, according to the Washington Post.
The art gallery owner rebuffed his offer, noted that it required documents proving ownership and referred the man to the FBI, according to prosecutors.
"Well-documented and known art is very hard to move once it has been stolen," said FBI Supervisory Special Agent Tim Carpenter.
Mafia exposed as artful dodgers "Gallery owners are our first line of defence in identifying pieces of art that do not have the appropriate documentation and should be brought to the attention of law enforcement."
Prosecutors asked a court on Thursday to approve the return of the painting to the family's estate to put up for auction.
The proceeds will be given to the insurance company that paid the theft claim and to charities supported by the family estate.
Statute of limitations laws on the robbery have expired, which means the suspected thief and the Maryland man will not face charges.
The suspected thief was previously convicted in federal court and served a prison term in connection with selling other stolen property, the agency said.
Alan Scott, an attorney for Mr Heller's estate and is the executor for Mrs Heller, told the Washington Post "of all the works that could have been recovered, this is the one that would have pleased them the most".