The Mindscape is the physical landscape of a person's mind.
The mindscape is first seen when Bill Cipher enters Stan's mindscape to get the combination to his vault for Gideon Gleeful. Dipper, Mabel and Soos enter to go defeat Bill and stop him from stealing the code to Stan's safe. The mindscape contains all of Stan's memories.
Bill mentions the mindscape while explaining to Dipper that he was stuck in there, being more or less a ghost that couldn't be seen or heard without a vessel to possess.
Years ago, while Ford was sleeping under a tree, he suddenly found himself in the mindscape, where he checked some files before finding Bill himself. After making a deal with him, building the universe portal and then having Fiddleford find out about Bill's true plan, Ford went back to the mindscape to see what was going on, getting a peek of the nightmare realm in the process.
In Weirdmageddon 3: Take Back The Falls, Bill enters Stan's mindscape once again as part of a deal, thinking it to be Ford's. Shortly after Stan revealed that he had switched personas with Ford in order to trick him, Ford, playing his part, uses the memory erasing gun to wipe Stan's mind, weakening Bill in the process. Stan proceeds to punch Bill as he melts and glitches, killing him.
The mindscape varies from person to person. For Stan, it centers around the Mystery Shack. For Ford originally, it was a vast, space-like place with hundreds of objects hovering aimlessly around it. In the present, it is a large wheat field that contains the broken Universe Portal, a swing set and the Stanowar. When a mind is being erased via the memory erasing gun, blue flames take over its mindscape, erasing every memory in it.