Inflammation is characterized by the infiltration of leukocytes from the circulation and into the inflamed area. Leukocytes are guided throughout this process by chemokines. These are basic proteins that interact with leukocytes to initiate their activation and extravasation via chemokine receptors. This is enabled through chemokine immobilization by glycosaminoglycans (GAGs) at the luminal endothelial surface of blood vessels. A specific stretch of basic amino acids on the chemokine, often at the C terminus, interacts with the negatively charged GAGs, which is considered an essential interaction for the chemokine function. Short-chain peptides based on this GAG-binding region of the chemokines CCL5, CXCL8, and CXCL12γ were synthesized using standard Fmoc chemistry. These peptides were found to bind to GAGs with high affinity, which translated into a reduction of leukocyte migration across a cultured human endothelial monolayer in response to chemokines. The leukocyte migration was inhibited upon removal of heparan sulfate from the endothelial surface and was found to reduce the ability of the chemokine and peptide to bind to endothelial cells in binding assays and to human rheumatoid arthritis tissue. The data suggest that the peptide competes with the wild-type chemokine for binding to GAGs such as HS and thereby reduces chemokine presentation and subsequent leukocyte migration. Furthermore, the lead peptide based on CXCL8 could reduce the disease severity and serum levels of the proinflammatory cytokine TNF-α in a murine Ag-induced arthritis model. Taken together, evidence is provided for interfering with the chemokine-GAG interaction as a relevant therapeutic approach.