Author: agowa338

Merging dictionaries within powershell

I needed to join dictionaries from within a list together, so that they become one whole dictionary.
First I thought it may be a trivial task. Than I used google and realized, that everything on the first page was just wrong or impracticable for lists with unknown size.

What I found, was stuff like:

$a = @{"foo"=1}
$b = @{"bar"=2}
$c = @($a,$b) | ForEach-Object {$_}

and

$a = @{"foo"=1}
$b = @{"bar"=2}
$c = $a + $b

The console output of $c will look identical for both:

Name                           Value
----                           -----
foo                            1
bar                            2

But the issue is with the data structure. This becomes a problem when we would like to access the data.
E.g.

$c['foo']

Will output $null for the first example and “2” for the 2nd. But why is this?
Lets convert it to json (without piping, this is important) to see:

ConvertTo-Json -InputObject $c

which outputs for the first case:

[
  {
    "foo": 1
  },
  {
    "bar": 2
  }
]

and for the 2nd what we originally expected:

{
  "bar": 2,
  "foo": 1
}

So now we know that the first one did not join the dictionaries, but warped them into a list. As we originally looked for a solution to join a list of dictionaries, this is not helpful.
(From now on $c is the output from the first example, as it is the same as my original input)
The 2nd one however looks helpful, but has two issues:

  1. It only works if we know how many elements we have (hard coded element count)
  2. It copies the list once for each element we want to add to it.

So let’s address these issues:

  1. For the 1st one something like this will do the trick: 
    $c | ForEach-Object -Begin {[Hashtable]$aa = @{}} -Process {$aa += $_} -End {$aa} | ConvertTo-Json
  2. But for the 2nd we need something a bit more advanced like 
    $c | ForEach-Object -Begin {[Hashtable]$aa = @{}} -Process {foreach($element in ($_.GetEnumerator())) {$aa.Add($element.Key,$element.Value)}} -End {$aa} | ConvertTo-Json

Twitter hack lessons learned

I’m not an twitter employee, but the recent “twitter hack” has shown some valuable lessons that everybody can learn from it.

* It is funny (and shocking) to read about how bad twitter’s security really is.
* And that TOTP-2FA cannot protect from social engineering (FIDO and/or smartcards would have)
* There is a need for “a secure line” e.g. company chat with different login for IT stuff to validate caller identity. Or to reset passwords only through an trusted (and authenticated) 3rd party (managers or hr). And the IT stuff should not simultaneously see who that 3rd party is, so that a phisher cannot simply continue to the next person.
* A 17 year old could gain access to twitter internal systems and was *not* caught because of audit logs of twitter.
* The concept “internal network” is very harmful. Systems need to be designed without the assumption of a “trusted/internal network”.
* There are always employees that fill out every phishing webpage.
* The system needs to be designed to have abuse detection.
* Inside threads are not only done by employees.
* Without an internal network there is no inside job.
* Don’t trust anyone not even your mom without validating her *and* her intend.
* Provide one and only one login portal for all of the companies portals and applications that share the same credentials. It should be a clear red flag that another url does not belong to those credentials even if the company logo is shown.
* BYOD is bad, provide all workers with secure (and usable!) notebooks (Not just remote workers, as it could be necessary to send your stuff home/remote without former notice, as covid has shown), Microsoft has a good writeup for that at https://docs.microsoft.com/en-us/windows-server/identity/securing-privileged-access/privileged-access-workstations
* Jumpservers and VPN don’t improve security.
* There should be a corporate policy that enforces a half annual pentest *and* mandates that *all* findings are addressed before the next one.
* And finally also mandate a brute force yourself where you check all of your accounts against known password lists, check new passwords against those lists before they are committing *and* block the pattern “$Word$Number$Specialchar” esp. “$Company$CurrentYear$Specialchar”, “$Month$CurrentYear$Specailchar”, …

MSSQL on kubernetes (non azure)

1. Copy the code below into a file named mssql.yaml
2. Apply the deployment file to your kubernetes cluster using:

sed -i s/TXlDMG05bCZ4UEBzc3cwcmQ=/$(pwgen -s 120 1 | base64 -w 0)/ mssql.yaml;
kubectl apply -f mssql.yaml

Note: username and password are base64 encoded and NOT encrypted.
Therefore do not store your credentials this way in a production environment, use 

read -sep "Enter mssql sa password: " mssql_sa_pass; kubectl create secret generic mssql2 --from-literal=password=$mssql_sa_pass --type=kubernetes.io/basic-auth

instead of adding it to the deployment yaml file.

apiVersion: v1
data:
  username: c2EK
  password: TXlDMG05bCZ4UEBzc3cwcmQ=
kind: Secret
metadata:
  name: mssql
type: kubernetes.io/basic-auth
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mssql-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mssql
  template:
    metadata:
      labels:
        app: mssql
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: mssql
        image: mcr.microsoft.com/mssql/server:latest
        ports:
        - containerPort: 1433
        env:
        - name: MSSQL_PID
          value: "Developer"
        - name: ACCEPT_EULA
          value: "Y"
        - name: SA_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mssql
              key: password 
        volumeMounts:
        - name: mssql
          mountPath: /var/opt/mssql
      volumes:
      - name: mssql
        hostPath:
          path: /srv/mssql
          type: DirectoryOrCreate
      initContainers:
      - name: install
        image: busybox
        command:
        - chown
        - "10001:10001"
        - "/work-dir"
        volumeMounts:
        - name: mssql
          mountPath: "/work-dir"
---
apiVersion: v1
kind: Service
metadata:
  name: mssql-deployment
spec:
  selector:
    app: mssql
  ports:
    - protocol: TCP
      port: 1433
      targetPort: 1433
  type: LoadBalancer

shadow compile error

when compiling shadow there is an error in the cross linux from scratch documentation.
instead of

echo "#define ENABLE_SUBUIDS 1" >> config.h

one has to run

echo "#define ENABLE_SUBIDS 1" >> config.h

as otherwise make fails with:

usermod.c:1364:10: error: 'vflg' undeclared (first use in this function)
  if (   (vflg || Vflg)
          ^
usermod.c:1364:10: note: each undeclared identifier is reported only once for each function it appears in
usermod.c:1364:18: error: 'Vflg' undeclared (first use in this function)
  if (   (vflg || Vflg)
                  ^
usermod.c:1365:10: error: 'is_sub_uid' undeclared (first use in this function)
      && !is_sub_uid) {
          ^
usermod.c:1372:10: error: 'wflg' undeclared (first use in this function)
  if (   (wflg || Wflg)
          ^
usermod.c:1372:18: error: 'Wflg' undeclared (first use in this function)
  if (   (wflg || Wflg)
                  ^
usermod.c:1373:10: error: 'is_sub_gid' undeclared (first use in this function)
      && !is_sub_gid) {
          ^
make[2]: *** [usermod.o] Error 1
make[2]: Leaving directory `/mnt/clfs/sources/shadow-4.2.1/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/mnt/clfs/sources/shadow-4.2.1'
make: *** [all] Error 2

gcc compile error

when compiling gcc 4.8 for cross linux from scratch it throws this error:

gcc-4.8.2/gcc/doc/gcc.texi:88: warning: @tex should only appear at the beginning of a line
gcc-4.8.2/gcc/doc/gcc.texi:208: no matching `@end tex'
gcc-4.8.2/gcc/doc/gcc.texi:208: no matching `@end multitable'
gcc-4.8.2/gcc/doc/gcc.texi:208: no matching `@end titlepage'

this can be fixed using this patch: https://osmocom.org/attachments/download/2798/patch-gcc46-texi.diff

and later on when compiling gcc for the 2nd time it throws:

In file included from ../../gcc-4.8.3/gcc/cp/except.c:1008:
cfns.gperf:101:1: error: 'const char* libc_name_p(const char*, unsigned int)' redeclared inline with 'gnu_inline' attribute
cfns.gperf:26:14: note: 'const char* libc_name_p(const char*, unsigned int)' previously declared here
cfns.gperf:26:14: warning: inline function 'const char* libc_name_p(const char*, unsigned int)' used but never defined
make[2]: *** [Makefile:1059: cp/except.o] Error 1
make[2]: Leaving directory '/media/clfs/sources/gcc-build/gcc'
make[1]: *** [Makefile:3909: all-gcc] Error 2
make[1]: Leaving directory '/media/clfs/sources/gcc-build'
make: *** [Makefile:858: all] Error 2

this can be fixed by applying this patch:
https://gcc.gnu.org/git/?p=gcc.git;a=patch;h=ec1cc0263f156f70693a62cf17b254a0029f4852

ncurses5.9 compile error

When building cross linux from scratch ncurses 5.9 does fail to compile with:

gcc -DHAVE_CONFIG_H -I../ncurses -I.  -D_GNU_SOURCE -DNDEBUG -I. -I../include -I/foo/include/ncurses -O2 --param max-inline-insns-single=1200 -c ../ncurses/lib_gen.c -o ../objects/lib_gen.o
In file included from ../ncurses/curses.priv.h:283,
                 from ../ncurses/lib_gen.c:19:
_823.c:835:15: error: expected ‘)’ before ‘int’
../include/curses.h:1594:56: note: in definition of macro ‘mouse_trafo’
 #define mouse_trafo(y,x,to_screen) wmouse_trafo(stdscr,y,x,to_screen)
                                                        ^
make[1]: *** [Makefile:785: ../objects/lib_gen.o] Error 1
make[1]: Leaving directory '/tmp/ncurses-5.9/ncurses'
make: *** [Makefile:111: all] Error 2

Use ncurses 6.1 instead.

m4 1.4.17 fails to compile

http://www.clfs.org/view/CLFS-3.0.0-SYSTEMD/mips64-64/cross-tools/m4.html

freadahead.c: In function 'freadahead':
freadahead.c:91:3: error: #error "Please port gnulib freadahead.c to your platform! Look at the definition of fflush, fread, ungetc on your system, then report this to bug-gnulib."
  #error "Please port gnulib freadahead.c to your platform! Look at the definition of fflush, fread, ungetc on your system, then report this to bug-gnulib."
   ^~~~~
make[3]: *** [Makefile:1837: freadahead.o] Error 1
make[3]: Leaving directory '/tmp/m4-1.4.17/lib'
make[2]: *** [Makefile:1602: all] Error 2
make[2]: Leaving directory '/tmp/m4-1.4.17/lib'
make[1]: *** [Makefile:1506: all-recursive] Error 1
make[1]: Leaving directory '/tmp/m4-1.4.17'
make: *** [Makefile:1461: all] Error 2

m4 does not compile with newer versions of gcc, therefore this patch is needed (thanks to stack overflow for linking to the 1.4.18 version, but as clfs uses 1.4.17 here is the port:

diff -ur m4-1.4.17/lib/fflush.c m4-1.4.17-patch/lib/fflush.c
--- m4-1.4.17/lib/fflush.c      2013-09-22 08:15:20.000000000 +0200
+++ m4-1.4.17-patch/lib/fflush.c        2020-02-04 17:23:47.964372943 +0100
@@ -33,7 +33,7 @@
 #undef fflush


-#if defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
+#if defined _IO_EOF_SEEN || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */

 /* Clear the stream's ungetc buffer, preserving the value of ftello (fp).  */
 static void
@@ -71,7 +71,7 @@

 #endif

-#if ! (defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */)
+#if ! (defined _IO_EOF_SEEN || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */)

 # if (defined __sferror || defined __DragonFly__) && defined __SNPT /* FreeBSD, NetBSD, OpenBSD, DragonFly, Mac OS X, Cygwin */      
@@ -145,7 +145,7 @@
   if (stream == NULL || ! freading (stream))
     return fflush (stream);

-#if defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
+#if defined _IO_EOF_SEEN || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */

   clear_ungetc_buffer_preserving_position (stream);

diff -ur m4-1.4.17/lib/fpurge.c m4-1.4.17-patch/lib/fpurge.c
--- m4-1.4.17/lib/fpurge.c      2013-09-22 08:15:20.000000000 +0200
+++ m4-1.4.17-patch/lib/fpurge.c        2020-02-04 17:23:47.964372943 +0100
@@ -61,7 +61,7 @@
   /* Most systems provide FILE as a struct and the necessary bitmask in
      <stdio.h>, because they need it for implementing getc() and putc() as
      fast macros.  */
-# if defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
+# if defined _IO_EOF_SEEN || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
   fp->_IO_read_end = fp->_IO_read_ptr;
   fp->_IO_write_ptr = fp->_IO_write_base;
   /* Avoid memory leak when there is an active ungetc buffer.  */
diff -ur m4-1.4.17/lib/freadahead.c m4-1.4.17-patch/lib/freadahead.c
--- m4-1.4.17/lib/freadahead.c  2013-09-22 08:15:20.000000000 +0200
+++ m4-1.4.17-patch/lib/freadahead.c    2020-02-04 17:23:47.976373311 +0100
@@ -25,7 +25,7 @@
 size_t
 freadahead (FILE *fp)
 {
-#if defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
+#if defined _IO_EOF_SEEN || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
   if (fp->_IO_write_ptr > fp->_IO_write_base)
     return 0;
   return (fp->_IO_read_end - fp->_IO_read_ptr)
diff -ur m4-1.4.17/lib/freading.c m4-1.4.17-patch/lib/freading.c
--- m4-1.4.17/lib/freading.c    2013-09-22 08:15:20.000000000 +0200
+++ m4-1.4.17-patch/lib/freading.c      2020-02-04 17:23:47.972373188 +0100
@@ -31,7 +31,7 @@
   /* Most systems provide FILE as a struct and the necessary bitmask in
      <stdio.h>, because they need it for implementing getc() and putc() as
      fast macros.  */
-# if defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
+# if defined _IO_EOF_SEEN || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
   return ((fp->_flags & _IO_NO_WRITES) != 0
           || ((fp->_flags & (_IO_NO_READS | _IO_CURRENTLY_PUTTING)) == 0
               && fp->_IO_read_base != NULL));
diff -ur m4-1.4.17/lib/fseeko.c m4-1.4.17-patch/lib/fseeko.c
--- m4-1.4.17/lib/fseeko.c      2013-09-22 08:15:55.000000000 +0200
+++ m4-1.4.17-patch/lib/fseeko.c        2020-02-04 17:23:47.964372943 +0100
@@ -47,7 +47,7 @@
 #endif

   /* These tests are based on fpurge.c.  */
-#if defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
+#if defined _IO_EOF_SEEN || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
   if (fp->_IO_read_end == fp->_IO_read_ptr
       && fp->_IO_write_ptr == fp->_IO_write_base
       && fp->_IO_save_base == NULL)
@@ -121,7 +121,7 @@
           return -1;
         }

-#if defined _IO_ftrylockfile || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
+#if defined _IO_EOF_SEEN || __GNU_LIBRARY__ == 1 /* GNU libc, BeOS, Haiku, Linux libc5 */
       fp->_flags &= ~_IO_EOF_SEEN;
       fp->_offset = pos;
 #elif defined __sferror || defined __DragonFly__ /* FreeBSD, NetBSD, OpenBSD, DragonFly, Mac OS X, Cygwin */
diff -ur m4-1.4.17/lib/stdio-impl.h m4-1.4.17-patch/lib/stdio-impl.h
--- m4-1.4.17/lib/stdio-impl.h  2013-09-22 08:20:02.000000000 +0200
+++ m4-1.4.17-patch/lib/stdio-impl.h    2020-02-04 17:23:47.964372943 +0100
@@ -18,6 +18,12 @@
    the same implementation of stdio extension API, except that some fields
    have different naming conventions, or their access requires some casts.  */

+/* Glibc 2.28 made _IO_IN_BACKUP private.  For now, work around this
+   problem by defining it ourselves.  FIXME: Do not rely on glibc
+   internals.  */
+#if !defined _IO_IN_BACKUP && defined _IO_EOF_SEEN
+# define _IO_IN_BACKUP 0x100
+#endif

 /* BSD stdio derived implementations.  */

Compile Linux Kernel for UBNT UNIFI

1. Download Cavium Open Source Distributions Octeaon SDK
Former: https://web.archive.org/web/20190215075110/https://github.com/Cavium-Open-Source-Distributions/OCTEON-SDK
Now: https://github.com/MarvellEmbeddedProcessors/Octeon-Toolchain
2. Clone latest linux kernel sources https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
3. Download the GPL Archive of UBNT (if they released it according to GPL and not infringed the license…), I was able to find it for the security gateway (UGW3) on the firmware download page by clicking on the row. However most other products it is not there.
4. 

tar -xjf toolchain-build-54.tar.bz2;
tar -xjf toolchain-GCC-7.3-build-230.tar.bz2

…to be continued…

WordPress nginx config

server {
listen 80;
listen [::]:80;
root "/var/www/vHOST";
server_name vHOST;
index index.html index.php;
client_max_body_size 2G;

rewrite ^/wp-json/.*$ index.php$uri last;

location ~ ^/.*/$ {
try_files $uri $uri/ /index.php?$uri;
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
location ~ \.php(/|$) {
include fastcgi.conf;
fastcgi_param SCRIPT_FILENAME "/var/www/vHOST/$fastcgi_script_name";
# With php7.0-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# With php7.0-fpm:
fastcgi_pass unix:/run/php/php7.0-fpm.sock;
}
}