The current Manor house was built in 1838, instead of a wooden manor house. The house has features of the classicism style, has nine lintels, is rectangular, plastered, has high foundations, impressive vaulted cellars.
The manor house was surrounded by a park of romanticism style. It is assumed that it was installed at the same time as the manor house was built.
For a long period of time the Manor belonged to the brothers Sirjatavičiai, who were descendants of the Bilevičiai family. In 1878 Skaraitiškė was passed to dukes Kontrimai family.
During the World War I the front line ran along Skaraitiškės and German soldiers stayed in the Manor. During the war, a fire broke out and 14 buildings of the Manor complex burned down.
Until 1940 the manor house was skilfully managed. Not only an 18th century portrait by an unknown artist of Ignotas Kontrim, the treasurer of the Grand Duchy of Lithuania, was kept here, but also antique furniture and rich manor library.
During the Soviet era, authorities nationalized all the buildings of the Manor and the owner M. Kontrimienė and her daughter Sofija were exiled to Siberia.
From 1995 the Manor belongs to the current owner. Currently a manor investment project is being prepared for renovation of buildings.