Here’s a traditional Japanese tea ceremony house. The small door is were you enter. ( I felt like a dog crawling in)
I guess the tea house was separated from the main house to isolate the tea ceremony and all that it means.
What you see can be found on big estates and for the most part belong to very wealthy people.