Solution:
a)
n^3+7(n^1..5)-3 and n^(log 8 -10(n^0.5)+7)
we already know how Big- O, Big-Omega, Theta works but here is an overview to go through once again,
By looking at functions we can observe that
n^3+7(n^1..5)-3 <= c1* ( n^(log 8 -10(n^0.5)+7))
and Please note that log2 8= log 2^3= 3 log 2= 3*1= 3, so n^log 8= n^3
n^3+7(n^1..5)-3 >= c2* (n^(log 8 -10(n^0.5)+7))
So we can say that
n^3+7(n^1..5)-3= 
(n^(log 8 -10(n^0.5)+7) )
Now,
n log n and n^1.1
Now we will compare by putting very large value of n and see the relation between these two
if n=2^256, then n log n= 2.9642775e+79 and
n^1.1= 5.8890708e+84
this means that n^1.1 is greater than n log n
Hence n log n= O(n^1.1).
5^n and 5.1^n
Only by looking at the function we can say that
5^n= o(5.1^n)
4^n and n!
We know that n! is almost equivalent to n^n
So here we can say that 4^n= o(n!)
b)
When we write generalized form of asymptotic notation then we only consider very large value of n, so the function which is taken at n>50 is going to be considered here,
f(n)= n log n + 100n
n log n + 100n <= c * (n log n )
for c= 2.
So, f(n)= O(n log n)
