Golang’s 10-year anniversary seems like a good time to roll out a Golang integration tutorial blog on how to validate addresses using the language and our Service Objects DOTS Address Validation US API. The development of this language started at Google but finished off in the open-source community. The intention behind the development of this language was to make it easier to code from an interpreted language and at the same time take advantage of compiled and strongly typed language features.
This language typically lands on the top ten lists of trending and demanded programming languages, and is becoming more and more popular with new generations of programmers. It is also platform agnostic, so you will be able to write this code on most machines. From what we have researched, it seems it was developed to replace C/C++ but it ended up replacing Python more. This can be debated but you can be the judge.
Regardless of how long you have been using Golang, whether it was from the beginning or more recently, we will have you up and validating addresses in no time. This tutorial will show you have to pull in a CSV file, iterate over the records while validating them and, finally, push the results to an output CSV file. This is just a simple “building blocks” type of example, and can be modified to suit your needs.
In this case, we used VS Code to do the development, and will be making comments and directions with respect to that IDE. We also used a file called GetBestMatchesExample.csv for our test data. You can reach out to us if you are interested in using this test data if you currently lack your own. It is 32 records long and tests a wide variety of responses from the service. The headers in the file contain BusinessName, Address, Address2, City, State and PostalCode, and are indexed from column A to F respectively.
With that said, let’s get started with the code. We start off needing to create two files, a .go file and a .json file. The .go file will contain all the code you are going to want to execute and the .json file will contain all the settings you will need. We are going to start with the .json file and then move through the .go file.
We named the .json file config.json. This will keep all the configuration we will need, and appears as follows:
{
"LicenseKey": "wsXX-XXXX-XXXX",
"ResultsFilename" : "ValidatedResults.csv",
"MAIN_ENDPOINT": "https://trial.serviceobjects.com/av3/api.svc/GetBestMatchesJson?",
"BACKUP_ENDPOINT": "https://trial.serviceobjects.com/av3/api.svc/GetBestMatchesJson?"
}
The LicenseKey element will contain the license key you received from Service Objects. License keys we distribute work for specific API’s, which means that if you get a license key for our Lead Validation product it will not work here. Another important note about the license keys is that only trial keys work on trial endpoints and only production keys work on production endpoints. Be sure to make the license key update when you go from the trial endpoint to the production endpoint.
MAIN_ENDPOINT and BACKUP_ENDPOINT, in this example, both point to the same trial environment. When you move to production those endpoints will change and switch to pointing at our production environments. Here they point to the same place, but in production the MAIN_ENDPOINT will instead point to the sws.serviceobjects.com endpoint and the BACKUP_ENDPOINT will point to the swsbackup.serviceobjects.com endpoint. Everything else will stay the same.
The ResultsFilename element is the name of the file we want to generate after validation. It will hold all the results of the addresses just validated through the API. This could have just as easily have been an input argument to the application instead. You can do whatever works for your business logic.
That’s it for the config.json file. The rest will happen in the .go file, which will pull these values into the code, as you will see.
We named the .go file AV3_GBM_GO.go. The GBM stands for GetBestMatches, which is the operation we will be calling in Address Validation US. This file will contain the library imports that we will be using throughout the tutorial, then some structs defined for unmarshalling the data coming back from the service, and lastly, the main methods/functions executed by the application.
The libraries that we are going to use will help us with parsing the CSV, parsing JSON, printing to the screen (usually for debugging in our case), IO, making an HTTP GET call, encoding parameters, and string conversion. These you can see in the import section of code.
import ( "encoding/csv" "encoding/json" "fmt" "io/ioutil" "log" "net/http" "net/url" "os" "strconv" )
Next, we have several structs to talk about: Configuration, Response, Address, ResponseError and ErrorDetails. Configuration will hold all those settings we added in the config.json file. Any changes to the .json files will need to be updated here as well.
type Configuration struct {
LicenseKey string
MainEndpoint string
BackupEndpoint string
ResultsFilename string
}
When we unmarshall the response we get from calling the API, we will need to have some structs ready for that. To help figure out what structs are needed we can look at the developer guides. The developer guide will house a “Try it Now” section for each service that contains API details (structure and types and so on) about the main operation for the service. If you need the API details for another operation in the service, we have those located in the Service Reference section for that particular service.
Based on the JSON being returned by the service, we see that we will need a struct for holding the addresses, for holding one address, for holding an error and for holding error details. Let’s start with a good response from the service.
type Response struct {
Addresses []Address `json:"Addresses"`
IsCass bool `json:"IsCass"`
}
This struct will house an array of addresses and an IsCass value. A couple of questions that may come to mind are, why is the address an array and isn’t there one validation per lookup on the API? There is one validation per lookup, but in the case of Address Validation US multiple addresses may be returned. When an address is validated by the system and there isn’t enough information to determine which address is the correct address, we will return all valid addresses. For instance, trying to validate an address that is equally as probable to be a street address starting with 27 E as it is 27 W then we return both. This is a very rare case, but it can happen. As for the IsCass value, it is a value that indicates if the address validation that occurred met the high standards of quality set by the USPS.
Naturally, the next struct we will talk about is the Address struct. This will contain all the fields that will be coming back in the response. Some will want to abbreviate this struct to only the fields that their system cares about based on their business logic, but it is our strong recommendation to pull all the values back and at least put them in a database. We do not do that here, since this is a tutorial on how to call our services using Golang, but there are plenty of articles out there that walk you through it.
type Address struct {
Address1 string `json:"Address1"`
Address2 string `json:"Address2"`
City string `json:"City"`
State string `json:"State"`
Zip string `json:"Zip"`
IsResidential string `json:"IsResidential"`
DPV string `json:"DPV"`
DPVDesc string `json:"DPVDesc"`
DPVNotes string `json:"DPVNotes"`
DPVNotesDesc string `json:"DPVNotesDesc"`
Corrections string `json:"Corrections"`
CorrectionsDesc string `json:"CorrectionsDesc"`
BarcodeDigits string `json:"BarcodeDigits"`
CarrierRoute string `json:"CarrierRoute"`
CongressCode string `json:"CongressCode"`
CountyCode string `json:"CountyCode"`
CountyName string `json:"CountyName"`
FragmentHouse string `json:"FragmentHouse"`
FragmentPreDir string `json:"FragmentPreDir"`
FragmentStreet string `json:"FragmentStreet"`
FragmentSuffix string `json:"FragmentSuffix"`
FragmentPostDir string `json:"FragmentPostDir"`
FragmentUnit string `json:"FragmentUnit"`
Fragment string `json:"Fragment"`
FragmentPMBPrefix string `json:"FragmentPMBPrefix"`
FragmentPMBNumber string `json:"FragmentPMBNumber"`
}
We are not going to describe each of these fields here, as our developer guides do a great job of going through them. If you have any additional questions about them or how they should be handled, feel free to reach out to us and we will be happy to chat with you about them. We also have plenty of blogs that touch on different return fields.
The ResponseError and ErrorDetails structs will be needed when examining an error from the operation. Both of those can be seen here respectively.
type ResponseError struct {
Error ErrorDetails `json:"Error"`
}
type ErrorDetails struct {
Type string `json:"Type"`
TypeCode string `json:"TypeCode"`
Desc string `json:"Desc"`
DescCode string `json:"DescCode"`
}
The errors are also well documented in the developer guides. When the service has a major issue with an address, or other problems such as the license key being wrong or expired, the operation will return an error response.
With these objects defined, we can now move to the main function where we will spend most of our time. There are a couple of supporting functions along the way that we will pop in and out of for processing the response from the API and reading a CSV file.
What the main function is going to do is attempt to call the service on the main endpoint and then, if there are errors, call the failover endpoint, the backup endpoint to attempt the call to the API again. Now, lets look at this.
As you typically see, we start by processing any arguments set to the main function and set other variables. This demonstration has only one input argument, which is the file name. Then we have a few lines of code to open up the config.json and decode it into the Configuration struct we defined earlier. What follows that is setting the MAIN_ENDPOINT, BACKUP_ENDPOINT, LICENSE_KEY and the ResultsFileName variables from the Configuration.
var filename = os.Args[1]
file, _ := os.Open("conf.json")
defer file.Close()
decoder := json.NewDecoder(file)
configuration := Configuration{}
err := decoder.Decode(&configuration)
if err != nil {
fmt.Println("error:", err)
}
var MAIN_ENDPOINT = configuration.MainEndpoint
var BACKUP_ENDPOINT = configuration.BackupEndpoint
var LICENSE_KEY = configuration.LicenseKey
var ResultsFilename = configuration.ResultsFilename
This are the items we are using for this demo, but you may have more or less depending on your needs.
Next, we pull the records from the input CSV file, and also create the results CSV and initialize the CSV writer for it. Our results CSV file will be named ValidatedResults.csv. At this moment we also write our first line to the CSV file. The first line will contain the header row across the top of the results file, so we know what each column represents. Here we reprint the input and the output from the API operation. We use the pipe (“|”) character in between the input and output so we can quickly observe in the file where the input ends and where the output starts.
records := readCsvFile(filename)
csvFile, e := os.Create(ResultsFilename)
if e != nil {
log.Fatal(e)
}
csvwriter := csv.NewWriter(csvFile)
AddressArray := []string{"Business Name", "Address 1", "Address 2", "City", "State", "ZIP Code", "|", "Address 1", "Address 2", "City", "State", "Zip", "Is Residential", "DPV", "DPV Desc", "DPV Notes", "DPV Notes Desc", "Corrections", "Corrections Desc", "Barcode Digits", "Carrier Route", "Congress Code", "County Code", "County Name", "Fragment House", "Fragment Pre Dir", "Fragment Street", "Fragment Suffix", "Fragment Post Dir", "Fragment Unit", "Fragment", "Fragment PMB Prefix", "Fragment PMB Number", "Error Type Code", "Error Type Desc", "Error Desc Code", "Error Desc", "General Error"}
_ = csvwriter.Write(AddressArray)
Now we start into the loop to iterate over the addresses from the input file. We use a for loop to do this, but usually clients have more data than just a few handfuls of addresses, so in practice this loop would usually be designed in an asynchronous fashion. The for loop we create starts at the second row, indicated by records[1:], because the input data has a header row that we do not want to process. The first thing the loop does is set up the first URL to call based on the MAIN_ENDPOINT, followed by the needed parameters appended in the query string. Now we are ready to make the call to the service, so we do and then wait for a response.
for _, InAddress := range records[1:] {
SOurl := MAIN_ENDPOINT + "BusinessName=" + url.QueryEscape(InAddress[0]) + "&Address=" + url.QueryEscape(InAddress[1]) + "&Address2=" + url.QueryEscape(InAddress[2]) + "&City=" + url.QueryEscape(InAddress[3]) + "&State=" + url.QueryEscape(InAddress[4]) + "&PostalCode=" + url.QueryEscape(InAddress[5]) + "&LicenseKey=" + LICENSE_KEY
var responseObject Response
var UnMarshallingError, ReadBytesError error
var responseData []byte
FirstResponse, GetFirstError := http.Get(SOurl)
If no errors are produced in executing the call to the service, then we will attempt to unmarshall the body of the response into the responseObject. The response from the service can still be an error but this error would, if it occurred, would be an error with validation rather than an error with the process of calling the service. At this point, if errors occurred in the call to the service, and in unmarshalling there was a validation error of type code three (Service Objects Fatal error), then we would switch over to the failover call and call the BACKUP_ENDPOINT.
if GetFirstError == nil {
responseData, ReadBytesError = ioutil.ReadAll(FirstResponse.Body)
UnMarshallingError = json.Unmarshal(responseData, &responseObject)
}
if GetFirstError != nil || UnMarshallingError == nil || ReadBytesError == nil {
var responseErrorObject ResponseError
var UnMarshallingError2 = json.Unmarshal(responseData, &responseErrorObject)
if UnMarshallingError2 == nil {
TypeCode, StringToIntCastError := strconv.Atoi(responseErrorObject.Error.TypeCode)
First, let’s assume the call to the service was good. We created a function to build the new row, via an array of strings, needed to add to the CSV file that would include the inputs, input/output delimiter and the results.
func ProcessResponse(AV3Response Response, csvwriter *csv.Writer, InputAddress []string) {
AddressArray := []string{InputAddress[0], InputAddress[1], InputAddress[2], InputAddress[3], InputAddress[4], InputAddress[5], "|", AV3Response.Addresses[0].Address1, AV3Response.Addresses[0].Address2, AV3Response.Addresses[0].City, AV3Response.Addresses[0].State, AV3Response.Addresses[0].Zip, AV3Response.Addresses[0].IsResidential, AV3Response.Addresses[0].DPV, AV3Response.Addresses[0].DPVDesc, AV3Response.Addresses[0].DPVNotes, AV3Response.Addresses[0].DPVNotesDesc, AV3Response.Addresses[0].Corrections, AV3Response.Addresses[0].CorrectionsDesc, AV3Response.Addresses[0].BarcodeDigits, AV3Response.Addresses[0].CarrierRoute, AV3Response.Addresses[0].CongressCode, AV3Response.Addresses[0].CountyCode, AV3Response.Addresses[0].CountyName, AV3Response.Addresses[0].FragmentHouse, AV3Response.Addresses[0].FragmentPreDir, AV3Response.Addresses[0].FragmentStreet, AV3Response.Addresses[0].FragmentSuffix, AV3Response.Addresses[0].FragmentPostDir, AV3Response.Addresses[0].FragmentUnit, AV3Response.Addresses[0].Fragment, AV3Response.Addresses[0].FragmentPMBPrefix, AV3Response.Addresses[0].FragmentPMBNumber, "", "", "", "", ""}
_ = csvwriter.Write(AddressArray)
}
Now, let’s assume there was an error in the validation or anywhere else earlier in the code. As mentioned earlier, when it comes to the validation error, the only type that should send us to a failover call is of type three, the Service Objects Fatal error. This error or any other non-validation error should bring us to the failover code.
if StringToIntCastError != nil || TypeCode == 3 || UnMarshallingError2 != nil {
SOurl := BACKUP_ENDPOINT + "BusinessName=" + url.QueryEscape(InAddress[0]) + "&Address=" + url.QueryEscape(InAddress[1]) + "&Address2=" + url.QueryEscape(InAddress[2]) + "&City=" + url.QueryEscape(InAddress[3]) + "&State=" + url.QueryEscape(InAddress[4]) + "&PostalCode=" + url.QueryEscape(InAddress[5]) + "&LicenseKey=" + LICENSE_KEY
SecondResponse, GetSecondError := http.Get(SOurl)
if GetSecondError != nil {
ProcessGeneralError(csvwriter, InAddress, "Network Error")
continue
}
responseData, ReadBytesError2 := ioutil.ReadAll(SecondResponse.Body)
if ReadBytesError2 != nil {
ProcessGeneralError(csvwriter, InAddress, "Error Reading SecondResponse.Body")
continue
}
Once we detect a need to failover, we start by updating the SOurl variable to the BACKUP_ENDPOINT and making the call to the API with the same address inputs. We check for issues connecting to the service and then reading the response. With either of these things, if an error is detected we bail out of further processing the address and call the ProcessGeneralError function to write a new line to the CSV file with details about the error for this record and we move on to the next record in the loop. If everything is good at this point, we attempt to unmarshall the response into the responseObject variable. If the unmarshalling is successful we call the ProcessResponse function to add a new line to the CSV file with the results. If, however, there is an error with unmarshall, the response from the service goes into the responseErrorObject so that we can add an error line to the CSV file using the ProcessError function.
var responseObject Response
var UnMarshallingError3 = json.Unmarshal(responseData, &responseObject)
if UnMarshallingError3 != nil {
var responseErrorObject ResponseError
var UnMarshallingError4 = json.Unmarshal(responseData, &responseErrorObject)
if UnMarshallingError4 != nil {
ProcessGeneralError(csvwriter, InAddress, "Error unmarshalling the error response")
continue
} else {
ProcessError(responseErrorObject, csvwriter, InAddress)
}
} else {
ProcessResponse(responseObject, csvwriter, InAddress)
}
And that concludes the loop, so after the line is added to the CSV file we continue with the next address record in the array.
This was one simple integration to loop through a CSV list of address, producing an output file with the inputs plus the results from validation. Hopefully you found this helpful, but if you have any questions about implementation or deeper details about the service, please reach out and we’ll be happy to go over it with you. Even if you just want to go over best practices for the business logic you are trying to implement, we’re here to help. This demonstration can also be altered simply to work for our other validation services.
