Consistent with the hypothesis that transcriptional dysregulation plays a causative role in cellular aging, Bollati et al. report that DNA methylation diminishes with age in humans:
Decline in genomic DNA methylation through aging in a cohort of elderly subjects
Loss of genomic DNA methylation has been found in a variety of common human age-related diseases. Whether DNA methylation decreases over time as individuals age is unresolved. We measured DNA methylation in 1097 blood DNA samples from 718 elderly subjects between 55 and 92 years of age (1-3 samples/subjects), who have been repeatedly evaluated over an 8-year time span in the Boston area Normative Aging Study. DNA methylation was measured using quantitative PCR-Pyrosequencing analysis in Alu and LINE-1 repetitive elements, heavily methylated sequences with high representation throughout the human genome. Age at the visit was negatively associated with Alu element methylation (beta=-0.12 %5mC/year, p=0.0005). A weaker association was observed with LINE-1 elements (beta=-0.06 %5mC/year, p=0.049). We observed a significant decrease in average Alu methylation over time, with a -0.2 %5mC change (p=0.012) compared to blood samples collected up to 8 years earlier. The longitudinal decline in Alu methylation was linear and highly correlated with time since the first measurement (beta=-0.089 %5mC/year, p<0.0001). In contrast, average LINE-1 methylation did not vary over time [p=0.51]. Our results demonstrate a progressive loss of DNA methylation in repetitive elements dispersed throughout the genome.
Recall that DNA methylation is a repressive epigenetic mark: when so-called CpG islands in promoter DNA are methylated, transcription is silenced. Therefore, decreased methylation should result in increased transcription.
The authors focused on repetitive elements found at high copy throughout the genome, rather than individual protein-coding genes, but that’s not to say the findings won’t generalize to all methylated DNA. After all, methylation and other mechanisms of transcriptional silencing may have initially evolved to prevent transcription from Alu, LINE and other types of repetitive element, and they’re methylated by the same mechanisms as “regular” promoters — implying that if there’s a genome-wide decrease in methylation of these elements, there’s probably a concomitant decrease in methylation happening at epigenetically silenced promoters as well.
Ouroboros 
So, if I’m not mistaken, methylation is sort of preventing our genome from expressing certain parts (are these transposable elements?), which is good, since expressing these parts is bad. And with aging we get less effective in preventing these bad things from happening?
You have it right. Transposable elements aren’t the only thing that a cell might want to silence, however — there are many genes that might be toxic or harmful in specific cell types, or whose expression might interfere with the normal functioning of those cells. That type of gene would be silenced as well, in a cellular context-specific manner.