Question: "Is speaking in tongues evidence for having the Holy Spirit?"
Answer: There are three occasions in the book of Acts where speaking in tongues accompanied the receiving of the Holy Spirit—Acts 2:4, 10:44-46, and 19:6. However, these three occasions are the only places in the Bible where speaking in tongues is an evidence of receiving the Holy Spirit. Throughout the book of Acts, thousands of people believe in Jesus and nothing is said about them speaking in tongues (Acts 2:41, 8:5-25, 16:31-34, 21:20). Nowhere in the New Testament is it taught that speaking in tongues is the only evidence a person has received the Holy Spirit. In fact, the New Testament teaches the opposite. We are told that every believer in Christ has the Holy Spirit (Romans 8:9; 1 Corinthians 12:13; Ephesians 1:13-14), but not every believer speaks in tongues (1 Corinthians 12:29-31).
So, why was speaking in tongues the evidence of the Holy Spirit in those three passages in Acts? Acts 2 records the apostles being baptized in the Holy Spirit and empowered by Him to proclaim the gospel. The apostles were enabled to speak in other languages (tongues) so they could share the truth with people in their own languages. Acts 10 records the apostle Peter being sent to share the gospel with non-Jewish people. Peter and the other early Christians, being Jews, would have a hard time accepting Gentiles (non-Jewish people) into the church. God enabled the Gentiles to speak in tongues to demonstrate that they had received the same Holy Spirit the apostles had received (Acts 10:47, 11:17).
Acts 10:44-47 describes this: “While Peter was still speaking these words, the Holy Spirit came on all who heard the message. The circumcised believers who had come with Peter were astonished that the gift of the Holy Spirit had been poured out even on the Gentiles. For they heard them speaking in tongues and praising God. Then Peter said, 'Can anyone keep these people from being baptized with water? They have received the Holy Spirit just as we have.'“ Peter later refers back to this occasion as proof that God was indeed saving the Gentiles (Acts 15:7-11).
Speaking in tongues is nowhere presented as something all Christians should expect when they receive Jesus Christ as their Savior and are therefore baptized in the Holy Spirit. In fact, out of all the conversion accounts in the New Testament, only two record speaking in tongues in that context. Tongues was a miraculous gift that had a specific purpose for a specific time. It was not, and never has been, the only evidence of receiving the Holy Spirit.