• awareness
• change is inevitable
• change related to sharing
• pros and cons

From the simplest "func main" to a large service. What choices to consider when your project grows and how to keep it on track for future changes.
We begin with a small project and then evolve it. Along the way depending on the choices we make, the design will change. I'll refer to directory layout, not as structure but as design.

• growth story
• design, not structure
• focus on package depencencies

$ tree rebel rebel ├── go.mod └── main.go 0 directories, 2 files 1 // Command rebel generates rebelious statement. 2 package main 3 4 import "fmt" 5 6 func main() { 7 Shout() 8 } 9 10 // Shout writes a statement to os.Stdout 11 func Shout() { 12 fmt.Println("We are Rebelz!") 13 } 14

Let's kick off our project. We'll name it "rebel" and use the repository domain "github.com/preferit/rebel". We choose to create a command, ie. package main. The implications for our ability to evolve Pros Easy to share command with go install github.com/preferit/rebel@latest Cons You Cannot share any logic within it with others (including yourself), Go disallows the import of main packages API documentation is hidden when using e.g. go doc, as a result of (1)

• rebel command

You have no intention to share any logic, the command is for you alone. You are happy and code along the nice feature of randomizing rebelious statements.

  1 // Command rebel generates rebelious statement.
  2 package main
  3 
  4 import (
  5     "fmt"
  6     "math/rand"
  7 )
  8 
  9 func main() {
 10     Shout()
 11 }
 12 
 13 // Shout writes a statement to os.Stdout
 14 func Shout() {
 15     fmt.Println(RandValue(phrases))
 16 }
 17 
 18 // RandValue returns a random element of the given slice or the zero value.
 19 func RandValue[T any](slice []T) (v T) {
 20     if len(slice) == 0 {
 21         return
 22     }
 23     i := rand.Intn(len(slice))
 24     return slice[i]
 25 }
 26 
 27 var phrases = []string{
 28     "We are Rebelz!",
 29     "You may take our lives, but you will never take our freeeedooooom!",
 30     "We ain't gonna take it, No!, we ain't gonna take it!",
 31 }
 32 

$ tree rebel
rebel
├── go.mod
└── main.go

0 directories, 2 files

• rebel logic grows
• first share

How do you convert the current state, a command, into an importable package while also keeping the command. First attempt; keep command in root and create a package with logic to generate the phrases.

$ tree rebel
rebel
├── go.mod
├── main.go
└── phrase
    └── phrase.go

1 directory, 3 files

  1 // Command rebel generates rebelious statement.
  2 package main
  3 
  4 import (
  5     "os"
  6 
  7     "github.com/preferit/rebel/phrase"
  8 )
  9 
 10 func main() {
 11     phrase.Shout(os.Stdout)
 12 }
 13 

  1 package phrase
  2 
  3 import (
  4     "fmt"
  5     "io"
  6     "math/rand"
  7 )
  8 
  9 // Shout writes a statement to os.Stdout
 10 func Shout(w io.Writer) {
 11     fmt.Fprintln(w, RandValue(phrases))
 12 }
 13 
 14 // RandValue returns a random element of the given slice or the zero value.
 15 func RandValue[T any](slice []T) (v T) {
 16     if len(slice) == 0 {
 17         return
 18     }
 19     i := rand.Intn(len(slice))
 20     return slice[i]
 21 }
 22 
 23 var phrases = []string{
 24     "We are Rebelz!",
 25     "You may take our lives, but you will never take our freeeedooooom!",
 26     "We ain't gonna take it, No!, we ain't gonna take it!",
 27 }
 28 

• how to move
• first attempt

We now have multiple stakeholders depending on it and going forward they might get affected. Our goal is to be able to make changes as freely as possible without affecting the stakeholders. Before we release these changes can we improve the design?

$ tree rebel
rebel
├── go.mod
├── main.go
└── phrase
    ├── phrase.go
    └── phrase_test.go

1 directory, 4 files

  1 // Command rebel generates rebelious statement.
  2 package main
  3 
  4 import (
  5     "fmt"
  6     "os"
  7 
  8     "github.com/preferit/rebel/phrase"
  9 )
 10 
 11 func main() {
 12     phrase.Shout(os.Stdout)
 13     fmt.Println()
 14 }
 15 

  1 package phrase
  2 
  3 import (
  4     "fmt"
  5     "io"
  6     "math/rand"
  7 )
  8 
  9 // Shout writes a statement to os.Stdout
 10 func Shout(w io.Writer) {
 11     fmt.Fprint(w, randValue(phrases))
 12 }
 13 
 14 // randValue returns a random element of the given slice or the zero value.
 15 func randValue[T any](slice []T) (v T) {
 16     if len(slice) == 0 {
 17         return
 18     }
 19     i := rand.Intn(len(slice))
 20     return slice[i]
 21 }
 22 
 23 var phrases = []string{
 24     "We are Rebelz!",
 25     "You may take our lives, but you will never take our freeeedooooom!",
 26     "We ain't gonna take it, No!, we ain't gonna take it!",
 27 }
 28 

• multiple stakeholders
• simplify for export

  1 // Command rebel generates rebelious statement.
  2 package main
  3 
  4 import (
  5     "bytes"
  6     "fmt"
  7     "os"
  8 
  9     "github.com/preferit/rebel/phrase"
 10 )
 11 
 12 func main() {
 13     // cache of last statement
 14     cache := "/tmp/lastphrase"
 15     last, _ := os.ReadFile(cache)
 16     // ignore error, we don't care
 17     var buf bytes.Buffer
 18     for {
 19         buf.Reset()
 20         phrase.Shout(&buf)
 21         if bytes.Equal(last, buf.Bytes()) {
 22             continue
 23         }
 24         break
 25     }
 26     // save
 27     _ = os.WriteFile(cache, buf.Bytes(), 0644)
 28     fmt.Println(buf.String())
 29 }
 30 

  1 package phrase
  2 
  3 import (
  4     "fmt"
  5     "io"
  6     "math/rand"
  7 )
  8 
  9 // Shout writes a statement to os.Stdout
 10 func Shout(w io.Writer) {
 11     fmt.Fprint(w, randValue(phrases))
 12 }
 13 
 14 // randValue returns a random element of the given slice or the zero value.
 15 func randValue[T any](slice []T) (v T) {
 16     if len(slice) == 0 {
 17         return
 18     }
 19     i := rand.Intn(len(slice))
 20     return slice[i]
 21 }
 22 
 23 var phrases = []string{
 24     "We are Rebelz!",
 25     "You may take our lives, but you will never take our freeeedooooom!",
 26     "We ain't gonna take it, No!, we ain't gonna take it!",
 27 }
 28 

• one stakeholder feature

Shortly after, Lisa swings by your office asking if you could implement something that doesn't repeat the same phrase every day? But you just did, how do you now share it with Lisa? At this point you'll realize that the initial redesign with the added phrase package, may need to change and your own code updated. This is one of those deadends where you have to turn around and find another path.

We can explore ways to use the current design, e.g. put some caching mechanism into the phrase package, but that seems out of place. Also the caching for the command invocation locally on your computer may differ from what is available on the intranet site, you don't know at this point. If we backtrack to the time where Max and Lisa came onboard, could we have taken a different route that would avoid this scenario?

• double stakeholder feature
• rework?
• out of place
• backtrack...

Let's go through our reasoning and see if we can find the culprit before trying to solve it. Lisa and Max where not very specific about their needs just that they wanted to show a random phrase in different ways, in the terminal and on a webpage. We choose to extract the phrase generating logic for Lisa to use and told Max to use the command. Then we added more logic in the command, because it was You and Max that saw the problem. What Lisa wants now is the logic that is found in the command, but her current dependency is elsewhere.

Looking at the code more closely we placed func Shout inside the phrase package, why? phrases don't shout, rebels do. So if we want to keep func Shout in the package rebel And we want to share it with Lisa, how do we solve that?

$ tree rebel
rebel
├── go.mod
├── main.go
└── phrase
    ├── phrase.go
    └── phrase_test.go

1 directory, 4 files

  1 // Command rebel generates rebelious statement.
  2 package main
  3 
  4 import (
  5     "fmt"
  6     "os"
  7 
  8     "github.com/preferit/rebel/phrase"
  9 )
 10 
 11 func main() {
 12     phrase.Shout(os.Stdout)
 13     fmt.Println()
 14 }
 15 

  1 package phrase
  2 
  3 import (
  4     "fmt"
  5     "io"
  6     "math/rand"
  7 )
  8 
  9 // Shout writes a statement to os.Stdout
 10 func Shout(w io.Writer) {
 11     fmt.Fprint(w, randValue(phrases))
 12 }
 13 
 14 // randValue returns a random element of the given slice or the zero value.
 15 func randValue[T any](slice []T) (v T) {
 16     if len(slice) == 0 {
 17         return
 18     }
 19     i := rand.Intn(len(slice))
 20     return slice[i]
 21 }
 22 
 23 var phrases = []string{
 24     "We are Rebelz!",
 25     "You may take our lives, but you will never take our freeeedooooom!",
 26     "We ain't gonna take it, No!, we ain't gonna take it!",
 27 }
 28 

• reasoning
• who saw the problem
• phrases don't shout
• how to keep and share?

$ tree rebel
rebel
├── cmd
│   └── rebel
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
└── rebel_test.go

2 directories, 5 files

  1 // Command rebel generates rebelious statement.
  2 package main
  3 
  4 import (
  5     "fmt"
  6     "os"
  7 
  8     "github.com/preferit/rebel"
  9 )
 10 
 11 func main() {
 12     rebel.MinimizeRepetition = true
 13     rebel.Shout(os.Stdout)
 14     fmt.Println()
 15 }
 16 

  1 package rebel
  2 
  3 import (
  4     "io"
  5     "os"
  6 )
  7 
  8 // Shout writes a statement to os.Stdout. Affected by
  9 // MinimizeRepetition flag.
 10 func Shout(w io.Writer) {
 11     next := randValue(phrases)
 12 
 13     if MinimizeRepetition {
 14         cache := "/tmp/lastphrase"
 15         last, _ := os.ReadFile(cache)
 16         for {
 17             if string(last) == next {
 18                 next = randValue(phrases)
 19                 continue
 20             }
 21             break
 22         }
 23         _ = os.WriteFile(cache, []byte(next), 0644)
 24     }
 25     w.Write([]byte(next))
 26 }
 27 
 28 var MinimizeRepetition bool
 29 

This design, is as effortless as the first attempt at the decision point, but it makes it easier to evolve the rebel logic. It also forces you to design rebel features in such a way that they may be used by Lisa as well as Max.

With this design, if Lisa came along with the same request; you could easily tell her to set MinimizeRepetition to true.

• Lisa import import github.com/preferit/rebel
• Max install go install github.com/preferit/rebel/cmd/rebel@latest

compare #7

• compare

The project will grow and at some point Lisa comes by and says they are switching languages for the intranet implementation but they really want to have access to the logic of generating rebelious statements. Could you write a service that exposes it?
Easy peasy, but where to put it? we a few choices

• switch language
• easy service
• but where?

The project will grow and at some point Lisa comes by and says they are switching languages for the intranet implementation but they really want to have access to the logic of generating rebelious statements. Could you write a service that exposes it?
Easy peasy, but where to put it? we a few choices

• switch language
• easy service
• but where?

$ tree rebel
rebel
├── cmd
│   └── rebel
│       ├── main.go
│       └── service.go
├── go.mod
├── phrase.go
├── rebel.go
└── rebel_test.go

2 directories, 6 files

The project will grow and at some point Lisa comes by and says they are switching languages for the intranet implementation but they really want to have access to the logic of generating rebelious statements. Could you write a service that exposes it?
Easy peasy, but where to put it? we a few choices

• switch language
• easy service
• but where?

$ tree rebel
rebel
├── cmd
│   └── rebel
│       ├── main.go
│       └── service.go
├── go.mod
├── phrase.go
├── rebel.go
└── rebel_test.go

2 directories, 6 files

$ tree rebel
rebel
├── cmd
│   └── rebel
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service.go

2 directories, 6 files

The project will grow and at some point Lisa comes by and says they are switching languages for the intranet implementation but they really want to have access to the logic of generating rebelious statements. Could you write a service that exposes it?
Easy peasy, but where to put it? we a few choices

• switch language
• easy service
• but where?

$ tree rebel
rebel
├── cmd
│   └── rebel
│       ├── main.go
│       └── service.go
├── go.mod
├── phrase.go
├── rebel.go
└── rebel_test.go

2 directories, 6 files

$ tree rebel
rebel
├── cmd
│   └── rebel
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service.go

2 directories, 6 files

$ tree rebel
rebel
├── cmd
│   └── rebel
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

3 directories, 6 files

The project will grow and at some point Lisa comes by and says they are switching languages for the intranet implementation but they really want to have access to the logic of generating rebelious statements. Could you write a service that exposes it?
Easy peasy, but where to put it? we a few choices

• switch language
• easy service
• but where?

$ tree rebel
rebel
├── cmd
│   └── rebel
│       ├── main.go
│       └── service.go
├── go.mod
├── phrase.go
├── rebel.go
└── rebel_test.go

2 directories, 6 files

$ tree rebel
rebel
├── cmd
│   └── rebel
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service.go

2 directories, 6 files

$ tree rebel
rebel
├── cmd
│   └── rebel
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

3 directories, 6 files

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       ├── main.go
│       └── service.go
├── go.mod
├── phrase.go
├── rebel.go
└── rebel_test.go

3 directories, 7 files

The project will grow and at some point Lisa comes by and says they are switching languages for the intranet implementation but they really want to have access to the logic of generating rebelious statements. Could you write a service that exposes it?
Easy peasy, but where to put it? we a few choices

• switch language
• easy service
• but where?

$ tree rebel
rebel
├── cmd
│   └── rebel
│       ├── main.go
│       └── service.go
├── go.mod
├── phrase.go
├── rebel.go
└── rebel_test.go

2 directories, 6 files

$ tree rebel
rebel
├── cmd
│   └── rebel
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service.go

2 directories, 6 files

$ tree rebel
rebel
├── cmd
│   └── rebel
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

3 directories, 6 files

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       ├── main.go
│       └── service.go
├── go.mod
├── phrase.go
├── rebel.go
└── rebel_test.go

3 directories, 7 files

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       └── main.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

4 directories, 7 files

The service is up and running and you get an idea of adding a feature to package rebel for scanning the web for statements that the rebel can shout. It doesn't feel like the feature fits in the rebel package directly but will be used by it. Let's call the feature crawl, but where to put it?

• potential unrelated feature
• internal

The service is up and running and you get an idea of adding a feature to package rebel for scanning the web for statements that the rebel can shout. It doesn't feel like the feature fits in the rebel package directly but will be used by it. Let's call the feature crawl, but where to put it?

• potential unrelated feature
• internal

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       └── main.go
├── crawl.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

4 directories, 8 files

The service is up and running and you get an idea of adding a feature to package rebel for scanning the web for statements that the rebel can shout. It doesn't feel like the feature fits in the rebel package directly but will be used by it. Let's call the feature crawl, but where to put it?

• potential unrelated feature
• internal

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       └── main.go
├── crawl.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

4 directories, 8 files

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       └── main.go
├── crawl
│   └── crawl.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

5 directories, 8 files

The service is up and running and you get an idea of adding a feature to package rebel for scanning the web for statements that the rebel can shout. It doesn't feel like the feature fits in the rebel package directly but will be used by it. Let's call the feature crawl, but where to put it?

• potential unrelated feature
• internal

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       └── main.go
├── crawl.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

4 directories, 8 files

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       └── main.go
├── crawl
│   └── crawl.go
├── go.mod
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

5 directories, 8 files

$ tree rebel
rebel
├── cmd
│   ├── rebel
│   │   └── main.go
│   └── rebelsrv
│       └── main.go
├── go.mod
├── internal
│   └── crawl
│       └── crawl.go
├── phrase.go
├── rebel.go
├── rebel_test.go
└── service
    └── service.go

6 directories, 8 files

You've come a long way since that early func main. The service is growing as Lisa finds new features to add. At some point you need to bring in more people to work on the service side. Initially you might start working in the same repository but down the road it may make more sense to split the service into it's own. The latter is what we're interested in.

$ tree rebel rebel ├── cmd │   ├── rebel │   │   └── main.go │   └── rebelsrv │   └── main.go ├── go.mod ├── internal │   └── crawl │   └── crawl.go ├── phrase.go ├── rebel.go ├── rebel_test.go └── service └── service.go 6 directories, 8 files

The design for change requires awareness of where we might end up in the future and selecting a route that enables it as frictionless as possible. Ie. if we'd selected option 4 when adding the service, the move at the end would have been even easier at the expense of mixing service logic with command logic. Depending on the service this may be a good tradeof.

You see these layerd packages e.g. net/http and encoding/json