Search all Github Repositories for an Organization

gumbler is a script I wrote to search through git commits and introduced in the blog post “Searching Through Git Commits”. Recently I wanted to run Gumbler across all repositories for an organization, the steps are discussed below.

First, we need to grab a list of repositories for the ORG. This can be done using the API

1
2
3
curl "https://api.github.com/orgs/[ORG NAME]/repos?page=1&per_page=10000" > repos.json
curl "https://api.github.com/orgs/[ORG NAME]/repos?page=2&per_page=10000" >> repos.json
...

Note, the API limits the number of values returned so you will want to update the page count to make sure you get them all.

Next we iterate through each repository, clone it, and run gumbler across the repo. A simple Ruby script is given below. Note, ignore any repos that were forked as they aren’t specific to the orgnization.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
require 'json'

# specify repos.json as an argument
file = File.read(ARGV[0])

# parse the json
data_hash = JSON.parse(file)

# iterate each hash
data_hash.each do |hash|
  # ignore forked repos
  if !(hash["fork"])
      puts "|+| Testing #{hash["name"]}"
      
      # clone the project
      `git clone https://github.com/[ORG]/#{hash["name"]}.git`
      
      # Gumbler requires full directory paths
      `ruby ~/gumbler/gumbler.rb -s -p #{hash["name"]} ~/[ORG]/results/`

      sleep(3)
  end
end