A group of over 150 mathematicians from around the world has issued a warning to policymakers regarding the inflated claims about artificial intelligence's ability to solve complex mathematical problems. This cautionary message follows heightened excitement over AI advancements, particularly after a young individual claimed to have solved one of the famous Erdős problems using OpenAI’s ChatGPT. Additionally, OpenAI announced that its AI had disproved an 80-year-old conjecture related to Erdős, which they touted as a significant achievement. However, many mathematicians remain skeptical about these claims, leading to the creation of the 'Leiden Declaration on AI and Mathematics.' This 11-page document advises against accepting promotional narratives without scrutiny and stresses the importance of consulting experts in policy decisions. It also addresses the financial pressures on academics and raises concerns about AI's involvement in military and surveillance programs, misinformation, and the unauthorized use of scholarly work to train AI systems. The declaration calls for a careful examination of AI's role in mathematics and broader scientific research.