Built on the site of what is now Upton Hospital in Albert Street, it was one of the first workhouses to be opened under the 1834 Poor Law Amendment Act, and could accommodate 440 people. It was regarded as a model of its type and was visited by people from all over the country. A chapel was built in 1837 and an infirmary to take 80 patients was added in 1906.

People were sent to the workhouse for many reasons, such as destitution, physical or mental illness, or homelessness. They may stay there for a few days only, or for the rest of their lives.

Inmates were expected to work; the work was repetitive and soul-destroying. Typical tasks for the men could be corn grinding or breaking rocks into stones for roads. Jobs for the women included cooking, cleaning and laundry work. The Superintendent of Labour supervised the work.

The Eton Union Workhouse, popularly known as Albert House, underwent several name changes. In the 1930s it was known as the Slough Public Assistance Institution and was run by Buckinghamshire County Council. Emergency Medical Service beds were added in 1940. On Monday 5th July, 1948, responsibility was transferred to the new National Health Service, and the building was renamed Upton Hospital.